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Preface 


The  purpose  of  this  study  was  to  review  the  literature 
concerning  Estimate-at-Completion  (EAC)  models.  This  thesis 
identifies  the  studies  and  their  sources,  provides  a 
categorization  and  summary  of  the  studies,  and  analyzes  the 
studies  for  strengths  and  weaknesses  in  relation  to  the 
results.  The  research  establishes  a  foundation  or  starting 
point  from  which  future  research  can  expand  upon. 

I  would  like  to  thank  my  faculty  advisor,  Major  David 
Christensen,  for  his  advice,  patience,  and  assistance. 

Thanks  also  goes  to  Mr.  Richard  Antolini  for  being  a  reader 
for  this  thesis.  I  would  like  to  thank  my  wife,  Beverly, 
and  children,  Jason,  Amanda,  and  Jennifer  for  their 
motivation  and  patience  during  this  difficult  time. 


ii 


Table  of  Contents 

Page 

Preface .  ii 

List  of  Tables .  v 

Abstract .  vi 

I .  Introduction  .  1 

General  Issue  .  I 

Research  Goals  .  4 

First  Objective;  Identify  and  collect  .  .  6 

Second  Objective:  Categorize  and  Svunmarize  6 

Third  Objective:  Analysis  of  Research  .  .  6 

Fourth  Objective:  study  Results  .  7 

Limitation  of  Study  .  3 

II.  Description  of  EAC  Models,  Studies,  and  Other 

Sources . 9 

Forecasting  . .  11 

Past  Performance  Factors  .  14 

Holeman  Model  (1975:3)  16 

Lollar  Model  (1980:10)  IB 

Totaro  Model  U987:19)  19 

Blythe  1982  Study  (1982:14) . 19 

Blythe  1984  study  (1984:15)  .  21 

Cryer  and  Balthazer  study  (1986:17)  ....  22 

Wallender  Study  (1986:18)  .  23 

Price  Study  (1985:16) .  24 

Reidel  and  Chance  Study  (1989:20)  .  25 

Covach  (NAVSEA)  Study  (1981:11)  .  28 

Land  and  Preston  Study  (1980:9)  .  32 

Bright  and  Howard  Study  (1981:12)  .  34 

Regression  Analysis  .  .  36 

Karsch  Study  (1974:2)  .  38 

Heydinger  (SAMSO)  study  (  39 

other  Techniques  .  ........  41 

El-Sabban  study ( 1973 ; l)  .  41 

Time  Series  Analysis .  43 

fiusse  Model  (1977:5)  44 

Chacko  Study  (1981:13)  45 

Weida  Model  (1977:8)  45 

Summary .  47 


Page 

III.  Analysis  of  Research  Studies  .  48 

Evaluation  Criteria  .  48 

El-Sabban  Model  (1) .  53 

Karsch  Model  1974  (2) .  54 

Holeman  Model  (3) .  55 

Busse  Model  (5) .  56 

Heydinger  (SAMSO)  Study  (7) .  56 

Wieda  Model  (8) .  58 

Lollar  Model  (10) .  58 

Chacko  Approach  (13) .  59 

Blythe  1984  Study  (15) .  59 

Totaro  Model  (19) .  60 

Model/Approach  Summary .  61 

Analysis  of  Comparison  Studies . .  .  61 

Karsch  (1976)  study  (4) .  62 

Hayes  Study  (6) .  62 

Land  and  Preston  Study  (9) .  63 

Covach  (NAVSEA)  Study  (11)  .  . .  64 

Bright  and  Howard  Study  (12) .  65 

Blythe  (1982)  Study  (14) . 66 

Price  study  (16) .  67 

wal lender  Paper  (18) .  68 

Balthazar  and  Cryer  Study  (17)  ....  .  68 

Reidel  and  Chance  Study  (20) .  69 

Summary . . .  71 

IV.  Conclusions  and  Recommendations . 72 

First  Objective:  Identify  and  Collection  .  .  72 

Second  Objective:  Categorization  and 

Summarization . . .  73 

Third  Objective:  Analysis  of  Studies  ....  74 

Fourth  Objective:  Study  Results.  ......  76 

Sutamary . . .  78 

Recommendations  for  Future  Studies .  79 

Appendix  A:  Explanation  of  Cost/Schedule  Control 

systems  Criteria  (C/SCSC)  .  81 

Appendix  8:  Formulas.  .  . .  87 

Appendix  C:  EAC  Formulas .  89 

Bibliography.  .  . . 96 

Vita .  101 


List  of  Tables 

Table  Page 

1.  Summary  of  EAC  Studies  and  Models . 12 

2.  Summary  of  Reidel  and  Chance  Study . 27 

3.  Analysis  of  EAC  Models  and  Studies . 50 


V 


AFIT/GCA/LSY/91S-6 


Atistract 

This  research  derives  from  the  Performance  Measurement 
discipline  and  consists  of  a  comprehensive  analysis  of 
Estimate-At-Completion  (EAC)  studies  published  since  1973. 
The  EAC  studies  consisted  of  models,  comparison  studies,  and 
computer  analysis  programs  .'"'^he  studies  were  located 
through  the  Defense  Technical  Information  Center  (DTIC) ,  the 
Cost  Library  at  Wright-Patterson  Air  Force  Base > OH,  and 
professional  periodicals  or  contracted  research. 

4^Each  study  was  categorised  by  formula  type  and  described 
in  terms  of  methodology  and  conclusions.  Each  study  was 
evaluated  based  on  clarity,  documentation,  methodology,  and 
source.  The  description  and  evaluation  of  the  studies  are 
summarized  in  two  tables. 

^  >After  reviewing  the  studies  some  areas  were  found  to  be 
weak.  The  AFSC  formula  that  uses  weighted  percentages  of  .3 
SPI  and  .3  CPI,  is  not  supported  by  a  critical  review  of  the 
literature.  In  the  area  of  comparison  studies,  different 
past  performance  factor  formulas  have  been  compared  with 
respect  to  different  percent  completion  points,  type  of 
contract,  and  type  of  product.  A  summary  of  the  results  are 
provided  in  Chapter  II.  >Little  work  has  been  done  comparing 
regression  formulas  to  past  performance  factor  formulas.  An 
important  outcome  of  this  research  identified  the  scarcity 
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of  formal  EAC  theory  or  relevant  research  concerning  the 
underlying  causes  of  •'^Why"  certain  EAC  formulas  are  better 
predictors  of  performance. 


ESTIMATE-AT-COMPLETION  RESEARCH  - 


A  REVIEW  AND  EVALUATION 


<aeasr.aL-I,a,§.\iS 

The  purchase  of  weapons  used  by  the  military  to  defend 
our  nation  and  aid  other  countries  is  a  major  function  of 
the  Department  of  Defense  (000).  A  dynamic  and  critical 
part  of  this  acquisition  process  is  estimating  the  cost  of 
these  weapon  systems.  In  a  1990  keynote  speech  given  to  a 
Cost  Analysis  Symposium  by  the  former  Under  Secretary  of 
Defense  (Acquisition) «  John  Betti«  the  function  of  cost 
estimating  was  defined. 

Vour  products  are  cost  estimates.  Vcur  customers  are 
the  people  who  use  those  estimates  in  the  decision 
making  process.  ....  the  cost  estimates  are 
critical  ingredients  in  deciding  among  competing 
alternatives  and  in  establishing  fundamental 
priorities.  They  are  equally  important  in  monitoring 
the  execution  of  approved  programs. 


(4:3) 

OOD  cost  estimates  have  made  the  evening  news,  generated 
newspaper  emd  magazine  articles  and  shortened  civilian  and 
military  careers  such  as  in  the  Navy  A-12  program  ( 35: Do¬ 
ll)  .  Managing  government  programs  is  not  an  easy  road. 

Early  detection  of  problems  can  save  a  program  from  becoming 
current  news. 
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In  order  for  a  program  manager  to  successfully  accomplish 
a  project,  tradeoffs  between  schedule,  contract  progress 
(performance)  and  cost  must  be  managed  (34:12) .  Robert 
Tiong  stated  '"'on  time",  "within  budget"  and  "according  to 
specifications"  should  be  the  basic  objectives  of  the 
program  control  system  for  any  major  project"  (44:32).  When 
costs  start  to  increase  it  is  a  good  indicator  that  the 
program  and  the  internal  process  are  in  trouble  (4:4). 
Monitoring  the  program  cost  througu  the  cost  estimating 
process  can  provide  early  warning  to  the  program  manager  of 
potential  problems.  Early  detection  enables  the  program 
manager  to  address  the  problems  and  manage  cost,  schedule 
and  performance  before  they  become  nonrecoverable. 

An  example  of  the  importance  of  SAC  formulas  is  the  Navy 
h-lZ  program.  The  A*12  program  was  a  medium  stealth  bomber 
that  was  designed  to  replace  the  A-6  Intruder.  Concept 
formulation  started  in  1984  with  the  PSO  contract  awarded  to 
McDonald  Douglas  and  General  Dynamics  in  1988.  This  program 
was  terminated  for  default  on  Jan.  V,l99l  after  an 
investment  of  $3.1  Billion  (35:30).  A  report  for  the 
secretary  of  the  Navy  concerning  the  A- 12  program  indicated 
the  program  analyst  provided  cost  estimates  generated  by 
formulas  which  forecasted  a  sizeable  cost  growth  on  the 
contract.  According  to  the  report,  the  program  manager 
elected  not  to  use  the  estimates,  but  developed  a  cost 
estimate  that  was  lower  than  the  estimate  of  the  program 
analyst.  An  independent  review  of  the  contract  produced  an 


estimate  that  was  considerably  higher  than  the  low  estimate 
briefed  by  the  program  manager  to  the  Major  Aircraft  Review 
(MAR)  committee  (3:i2-l3) .  The  independent  estimate 
supported  the  cost  analyst's  position.  This  was  only  a  part 
of  the  problem,  but  the  final  result  made  headlines  with  the 
termination  of  the  A-12  program  and  resignation  of  high- 
ranking  civilian  and  DOD  personnel. 

The  Gramm-Rudman-Hol lings  Act,  ending  of  the  Cold  War, 
internal  economic  problems  in  the  United  States,  and  public 
opinion  factors  are  driving  Congress  to  balance  the  Federal 
budget.  The  budget  of  the  Department  of  Defense  (DOD)  has 
declined  for  the  last  six  years  (4:3).  The  DOD,  program 
managers  and  financial  managers  are  convinced  the  budget 
cuts  are  the  beginning  of  a  new  era.  John  Betti,  in  his 
speech  to  the  DOD  Cost  Analysis  Symposium,  enforced  this 
belief. 

The  costs  are  very  important  because  the  competition 
for  the  taxpayers'  dollars  is  becoming  ever  more 
fievce.  We  all  know  that  defense  budgets  are  getting 
tighter.  FY91  budget  will  likely  mark  the  sixth 
.straight  year  of  decline  in  real  dollars  for  the  DOD.  . 

.  .  .  There  is  little  rea.son  to  believe  that  we  should 
expect  more  liberal  defense  spending  in  the 
foreseeable  future. 

(4:3) 

In  order  to  meet  the  ever  tightening  budgets,  the  DOD 
needs  reliable  and  sometimes  frequent  cost  estimates  from 
which  to  prepare  and  defend  its  budgets  and  programs.  This 
is  not  only  true  at  the  Congressional  level  but  also  within 
the  DOD  itself,  at  the  Service  level  (Air  Force,  Navy,  etc.) 
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and  within  each  Service.  The  bottom  line  with  most  reviews 
is  "How  much  will  it  cost?"  The  estimate  of  the  forecasted 
final  contract  cost  is  termed  an  Estimate-at-Completion 
(EAC)  (22:113). 

The  decreasing  dollars  and  increasing  problems  from 

advanced  technology  (schedule  delays,  rework)  will  make  the 

job  of  program  manager  even  tougher.  General  Nauseef 

reiterated  this  idea  in  a  keynote  speech  to  the  1990 

Cost/Schedule  Control  System  Criteria  National  Workshop. 

At  the  same  time,  the  acquisition  of  major  systems  is 
more  complicated.  Cutbacks  are  impacting  program 
management  programs  with  cost  and  schedule  problems 

receivi~7  more  visibility . As  the  budget  gets 

smaller  ,  st  and  schedule  perturbations  get  bigger  and 
receive  more  attention.  Ever  programs  with  good  cost 
performance  will  be  caught  in  the  budget  changes. 

(36:25) 

Cost  estimating  is  now  in  the  spotlight  and  the  importance 
of  cost  estimating  to  a  program,  the  000  and  program 
managers,  is  at  a  critical  level. 


Res9arch__goals 

The  manager  (Government  and  Contractor)  of  a  program 
relies  upon  the  financial  community  to  provide  an  Estimats- 
at -Completion  (EAC) .  The  estimate  can  be  very  detailed  and 
take  months  to  finish  or  it  can  be  produced  in  a  short 
amount  of  time  by  using  formulas.  The  quick  turn  around 
time  the  DOO  expects  for  information  and  the  internal  time 
constraints  in  the  program  office  make  the  use  of  formulas 
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very  convenient.  This  is  the  area  of  focus  for  this 
research  —  Estimate-at-Completion  formulas. 

A  review  of  the  available  studies  over  the  last  20  years 
produced  some  general  observations  about  the  EAC  area.  The 
trend  in  recent  EAC  studies  has  been  a  comparative  type  of 
research.  This  type  of  research  uses  a  small  sample  of  EAC 
formulas,  collects  specific  data  with  respect  to  product  or 
acquisition  phase  (Research  &  Development,  Production) ,  and 
statistically  or  mathematically  evaluates  the  formulas  in 
order  to  pick  the  ''best”  predicting  formula.  There  seems  to 
be  no  basic  theory  developed  upon  which  the  studies  are 
based.  Seme  of  the  earlier  studies  dealing  with  the 
weighting  of  indices  (Schedule  and  cost)  showed  conflicting 
results  using  the  same  data  bases. 

Developmental  model  research  has  been  done  but  most  of 
the  models  are  poorly  documented  and  lack  validation.  In 
addition  there  has  been  little  follow-on  work  to  expand  or 
justify  the  formulas.  It  is  hard  to  do  development  or 
follow-on  research  when  you  do  not  know  what  has  already 
been  accomplished.  After  the  research  has  been 
accomplished,  distribution  to  the  users  in  the  EAC  field 
appears  to  be  limited  and  few  studies  are  published  in 
periodicals  or  professional  journals.  There  appears  to  be 
many  studies  that  have  been  accomplished,  read  by  one 
person,  then  placed  in  a  library  never  to  be  seen  again. 

A  theoretical  base  from  which  to  work  is  missing  in  the 
CAC  area.  The  studies  are  like  individual  building  blocks 
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vaiting  to  be  joined  into  a  foundation  for  future  expansion. 
Unless  the  base  is  defined  there  is  no  starting  point  for 
progressive  expansion  of  EAC  research.  This  can  result  in 
repetitive  and  poorly  designed  research  work  that  can 
stagnate  advancement  in  the  EAC  area. 

This  research  has  four  objectives  which  are  presented 
below. 

First  Objective!  Identify  and  Collect. 

In  order  to  know  what  has  been  accomplished,  Estimate- 
at-Comi  \etion  studies  and  sources  of  EAC  formulas  must  be 
identified  and  collected.  This  research  will  identify  and 
colJ ict  available  EAC  studies  and  sources. 

Second  _Qbiective.:_  .  Categorize  and  Summarize. 

After  collecting  the  research  sources,  they  will  be 
categorized  and  summarized  by  formula  type,  time  frame, 
source  type  (service  doing  fo»~aula) ,  and  the  database  used 
in  the  research.  This  will  show  what  study  trends  and  areas 
of  concentration  the  research  has  focused  upon. 

Third  Objective; . Analysis  of  Research. 

Once  summarized,  the  studies  will  be  analyzed  for 
strengths  and  weaknesses.  Methodology,  documentation,  data 
sampling,  clarity,  and  logic  behind  the  study  wiU  be 
reviewed.  The  analysis  should  provide  a  measurement  of  the 
confidence  that  can  be  applied  to  che  conclusions  of  the 
research . 
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Fourth  Objective:  Study  Results. 

The  final  objective  is  to  evaluate  the  conclusions  of  the 
research  based  upon  the  merits  of  the  studies.  This  will 
establish  a  starting  point  from  which  future  research  can 
build.  Establishing  the  historical  base  would  allow  for  a 
more  scientific  approach  to  EAC  research  rather  than 
"shooting  in  the  dark". 

As  this  review  looks  at  different  studies,  approaches, 
computer  analysis  programs,  classroom  instruction,  and 
models  there  is  one  common  thread  that  weaves  throughout  the 
studies.  The  formulas  incorporate  Cost/Schedule  Control 
System  Criteria  (C/SCSC)  data  element  terminology. 

Most  major  defense  contracts  are  required  to  submit  Cost 
Performance  Reports  (CPR)  or  Cost  Schedule  Status  Reports 
(C/SSR) .  For  contractors  submitting  CPRs,  their  system  must 
pass  an  evaluation  based  on  the  DOD  Cost/Schedule  Control 
Systems  Criteria  (C/SCSC) .  One  of  the  goals  of  C/SCSC  is  to 
provide  confidence  that  the  data  provided  are  accurate  and 
reliable  (16:1-1) .The  data  element  nomenclature  is  unique  to 
the  C/SCSC  area.  In  order  to  understand  the  terms  and 
assumptions  used  throughout  this  study,  a  brief  explanation 
of  C/SCSC,  related  performance  reports, and  terms  are 
provided  in  Appendix  A  with  the  mathematical  relationships 
presented  in  Appendix  B. 
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Limitation  of  Study 


The  research  sources  reviewed  represent  the  majority  of 
the  major  studies  and  sources  over  the  last  20  years.  This 
review  does  not  present  itself  as  an  all  inclusive  review  of 
all  the  EAC  literature  published  during  this  period. 

Certain  studies  have  been  lost  over  the  past  years  (John 
Sincavage;  TARISM)  and  some  studies,  exclusively  done  by 
each  Service  (AD  Study,  AFSC  Study  by  Wal lender)  are  buried 
in  cost  Libraries  across  the  nation.  Scarce  travel  funds 
and  time  constraints  prohibited  on  site  research  at  these 
libraries. 

The  following  chapters  will  provide  a  literature  review 
(Chapter  2),  critical  analysis  of  each  study  (Chapter  3), 
and  conclusions  about  the  research  objectives  and  future 
research  recommendations  (Chapter  4) . 
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II.  Description  of  EAC  Models.  Studies,  and  Other  Sources 


In  Chapter  I  an  Estimate-at-Completion  (EAC)  was  defined 
as  the  forecasted  final  contract  cost.  So  what  should  this 
EAC  contain?  In  one  of  the  books  written  on  C/SCSC,  Quentin 
Fleming  stated  there  are  four  factors  a  contractor  must 
consider  when  developing  an  EAC: 

1.  The  performance  to  date,  BCWP,  as  related  to 
the  original  plan,  BCWS.  This  would  involve 
evaluating  the  schedule  performance  such  as  a 
Schedule  Performance  Index  does  (Appendix  A) . 

2.  The  actual  costs  to  date,  ACWP.  This  would 
include  direct  costs  as  well  as  indirect 
costs . 

3.  A  projection  of  future  performance.  A  look  at 
future  changes,  additional  tasks,  problem 

areas,  and  uncertainties  or  risk  in 
accomplishing  the  tasks. 

4.  An  estimate  of  the  cost  of  the  remaining  work 
on  the  contract. 


(22:114) 


Whether  a  government  cost  analyst  or  a  contractor  is 
developing  or  analyzing  an  EAC,  these  factors  should  be 
present  in  order  to  have  a  comprehensive  and  meaningful  EAC. 
The  above  factors  are  also  evaluated  in  the  POP  Joint 
Implementation  Guide  (JIG),  Appendix  E,  Section  IV 
(Analysis)  criterion  6,  with  19  questions  focusing  on  the 
contractor  development  of  an  EAC  (16;  Appendix  E.IV.6:  E- 
13-14) . 

Considering  the  above  factors  Slemaker  takes  the 
definition  of  EAC  to  a  finer  level.  He  defines  an  EAC  as  an 
estimate  of  the  actual  work  left  plus  the  actual  costs  of 
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the  work  performed  (ACWP)  to  date.  This  can  be  represented 
mathematically  in  the  following  formula: 

EAC  =  CACWP  +[[BAC-CBCWP]  X  PPP]  (1) 

Where  CACWP  refers  to  the  ciomulative  cost  to  date  and  CBCWP 
is  the  cumulative  '•Eamed  Value"  of  the  work  actually 
accomplished.  The  difference  between  the  BAC  (Total  Work) 
and  the  CBCWP  is  the  work  remaining.  A  Past  Performance 
Factor  (PPF)  is  a  factor  that  indicates  how  the  contractor 
has  performed  to  date.  The  work  remaining  (in  dollars) 
multiplied  by  the  PPF  will  give  the  expected  cost  of  the 
work  remaining.  The  actual  cumulative  cost  and  the  estimate 
of  work  remaining  will  produce  an  EAC  (43:205-206). 

There  are  two  different  methods  used  to  formulate  EACs: 
analytical  and  mathematical  (43:205,  22:116-117).  The 
analytical  method  is  basically  re-estimating  the  rest  of  the 
work  from  the  lowest  level  of  work  tasks.  This  method  is 
very  time  consuming  with  detailed  calculations  and  intensive 
management  reviews.  The  Joint  Implementation  Guide  requires 
this  type  of  estimate,  a  "grassroots"  EAC,  annually.  The 
mathematical  method  requires  the  use  of  CPR  or  C/SSR  data 
elements  such  as  cumulative  ACWP  (actual  cost),  BAC, and  BCWP 
(earned  value)  (43:208,  22:181). 

A  true  analytical  review  or  comprehensive  EAC  can  only 
be  performed  by  the  contractor.  He  is  the  one  that  knows 
how  he  does  the  work,  what  tasks  he  must  be  done,  and  the 
internal  ins  and  outs  of  performance  (16:3-17).  Because  of 
time  constraints,  travel  expense,  and  manpower  limitations 
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the  government  relies  upon  the  mathematical  method  for  doing 
an  EAC.  Since  a  validated  system  assumes  the  data  are 
reliable,  data  elements  of  the  CPR  are  used  to  calculate  an 
EAC. 


Forecasting 

Forecasting  has  been  defined  as  "a  process  which  has  as 
its  objective  the  prediction  of  future  events  or  conditions" 
(31:3).  EAC  formulas  predict  a  future  event  -  the  final 
cost.  The  EAC  literature  can  be  divided  into  three  general 
categories  of  formulas  or  approaches  for  forecasting  an  EAC. 

1.  Indices  or  past  performance  factors 

2.  Regression  equations 

3 .  Other 

Table  1  provides  a  general  summary  of  the  models  and 
studies  reviewed  in  this  chapter.  A  "Study"  is  defined  as 
research  that  compares  different  EAC  formulas  using 
Cost/Schedule  performance  data.  "Models"  present  EAC 
formulas  as  single  equations  or  a  sequence  of  steps 
ortechniques  to  produce  an  EAC,  but  reference  limited  (one 
to  fiv'<3  samples)  or  no  empirical  work  supporting  the 
formulas.  An  explanation  of  each  column  in  Table  1  is 
provided  below: 

Column 

Heading  Explanation 

Ref.  #  Provides  a  reference  number  for  each 
study.  This  number  will  be  used  as  a 
cross  reference  throughout  this  thesis. 
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TABLE  1 


SUMMARY  OFEAC  STUDIES  AND  MODELS 


s 

Author 

Year 

EAC  Formulas 

Type  of 
Contract 

SER 

ST 

INDEX 

LESS. 

OTHER 

Iggl 

WT 

WSM 

L 

NL 

0 

R&D 

PROD 

n 

El-Sabban 

X 

B 

M 

m 

1974 

X 

ESSt 

AF 

M.C 

B 

Holeman 

1975 

mat 

X 

B 

M 

Kl 

Kaisch 

1976 

X 

ESBt 

msst 

ca 

f.cm 

m 

Busse 

1977 

X 

Bin 

AF 

M 

B 

1977 

X 

WEM 

X 

mm 

AF 

cf 

Ll 

Heydinger 

1977 

X 

B 

X 

Bn 

AF 

M.C 

Kl 

Weida 

1977 

X 

X(22) 

AF 

M 

9 

Land 

Preston 

1980 

X 

■ 

■ 

■ 

■ 

X 

X* 

X* 

AF 

HB 

m 

Lollar 

1980 

X 

B 

AF 

M 

m 

Covach 

1981 

X 

X 

X 

X 

B 

X(14) 

Bn 

N 

C.M 

12 

Bright 

Howard 

1981 

X 

X 

X 

H 

X 

X 

n 

■ 

A 

M.C 

B 

Chacko 

1981 

X 

u 

M 

Kl 

Blythe 

1982 

X 

X 

X 

X(19)S 

AF 

C.M.F 

m 

Blythe 

1984 

X 

X(7)S 

X(19)S 

.AF 

M.F 

B 

Price 

1985 

■a 

B 

Xf57) 

AF 

C 

17 

Cryer 

Balthazer 

1986 

■ 

■ 

■ 

H 

H 

■ 

■ 

H 

Xfl9)S 

AF 

wsst 

in 

Wallendcr 

1986 

B 

AF 

F 

B 

Toiaro 

1987 

B 

mi3 

ca 

M 

L 

Reidel 

Chance 

1989 

X 

■ 

IS 

H 

X 

■ 

■ 

X(16) 

X(40) 

af 

F.C 

Acronym  Defmiiton: 

A*  Army 

AF « Air  Force 

C  •  Comparative  Study 

CPI  •  Cost  Performance  Index 

F  -  Fbllow-on  Study 

I • Independent 

L  •  Linear  Single  Variable  Model 
M  •  Model/Approach 
N  *  Navy 

•  -  30Total  Programs  (no  breakout) 


NL  •  Non  Linear  Single  Variable  Model 

O  •  Other  Methods 

PROD  •  Production  Contracu 

RdiO  ’  Research  and  Devetopmeni  Contracu 

S  *  Same  Database 

SCI  •  Schedule  Index  x  Cost  Index 

SER  •  Armed  Services 

SPI  •  Schedule  Perfonnaitce  Index 

ST '  Ssudy  Type 

WT  •  Weighted  Per  Cent  for  Cost  and  Schedule  Indices 
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Author 


Self  explanatory. 
Self  explanatory. 


Year* 

CPI  Formulas  using  the  Cost  Performance 

Index. 

SPI  Formulas  using  a  Schedule  Performance  Index. 

SCI  Formulas  using  the  CPI  multiplied  by  the 

SPI. 

WT  Formulas  using  a  weighting  for  Schedule 

and  Cost  indices  added  together:  W  ^  SPI  -»• 

W  2  CPI  =*  1. 

o  Other  methods  or  combinations  in  a  category. 

L  Refers  to  linear  regression  models,  with  the 

number  of  independent  variables  in  parentheses. 


NL  Non  linear  regression  models  with  the  number  of 

independent  variables  in  parentheses. 

Type  of  contract  the  study  or  model  used  for 
illustration  or  comparison.  RSiO  refers  to 
Research  and  Development  and  engineering  and 
Material  (formally  FSD)  contracts.  PROD 
refers  to  Production  contracts. 

S£R  The  branch  of  the  Armed  Services  that 

contracted  or  performed  the  study. 

ST  Indicates  the  different  structures  used  in 

the  studies. 

C  •  A  study  that  compares  different  formulas. 

M  -  A  model  or  approach  is  developed  or 
modified. 

F  -  Further  research  Is  done  using  a  previous 
study  as  its  base. 

Each  study  and  model  is  categorized  based  upon  the  £AC 
formulas  within  the  research  study  or  sources. 

The  remainder  of  the  chapter  describes  each  model  and 
study  listed  in  Table  1.  Each  study  or  model  subheading 
includes  the  year  of  the  study  and  the  reference  number  from 
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Table  1.  Past  Performance  Factors  (PPF)  research  is 
addressed  first  with  regression  analysis  formulas  next.  The 
last  category  is  Other  which  will  look  at  research  into 
different  techniques.  For  each  model  or  study,  the 
equations,  evaluation  criteria,  and  results  (if  appropriate) 
are  described. 

Past  Performance  _EactQr3 

Past  Performance  Factors  (PPF)  are  ratios  of  BCWS,  BCWP, 
and  ACWP,  CPI,  SPI,  CV%,  and  SV%  are  the  common  ratios  and 
are  used  with  current  month,  moving  average  month  or 
cumulative  data.  The  “factors"  can  be  ratios  used  alone  or 
in  combinations  in  order  to  produce  an  EAC.  The 
mathematical  formulas  for  the  above  ratios  are  presented  in 
Appendix  B. 

The  two  primary  indices  are  the  CPI  and  SPI.  The 
stability  of  the  CPI  was  the  focus  of  an  Air  Force  Institute 
of  Technology  (AFIT)  1990  Masters'  Thesis  by  Kirk  Payne.  He 
evaluated  the  CPI  stability  (within  +  or  -  10%)  at  the  S0% 
completion  point  of  a  contract  (39:1-2).  The  CPI  was  shown 
to  be  stable  at  50%  and  lower  points  of  completion  depending 
upon  which  method  was  used  (39:22-25).  So  as  the  contract 
progresses  the  Cumulative  CPI  is  an  increasingly  stable 
indicator  of  performance. 

As  stated  above,  the  indices  can  represent  different  t.'me 
periods.  Moving  averages  are  averaged  data  for  whatever 
time  period  is  chosen.  EAC  formulas  usually  use  three,  six 
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and  12  months  as  time  periods.  The  averaging  method  uses 
two  techniques;  Average  of  Ratios,  and  Ratio  of  Sums.  The 
Average  of  Ratios  technique  calculates  each  monthly  index 
then  divides  by  the  number  of  months. 

CPX.  S  S  (BCWP  /  ACWP)  /  n  (2) 

SPZ.  s  £  (BCVP  /  BCWS)  /  n  (3) 

Where 

n  a  number  of  months 

This  method  will  be  indicated  in  this  study  by  the  index 
name  with  the  period  of  time  as  a  subscript  (CPI  ,  ,SPI  ). 

The  second  method  is  a  Ratio  of  Sums  of  monthly  data 
through  the  chosen  time  period. 

'CPX  ,  a  £  BCVP  /  £  ACVP  (4) 

'3PX  ,  a  E  BCVP  /  £  SCV8  (5) 

This  method  will  be  indicated  by  a  bar  over  the  index  with 
the  subscript  indicating  the  period  of  time  (CPI  3,SPI  ) 
{X2;23-24) . 

Combinations  of  the  Indices  are  also  used  to  produce 
performance  factors.  The  CPI  and  SPI  may  be  either 
multiplied  or  added  together.  Their  product  is  sometimes 
termed  the  ''ScheduIe'<!ost’*Index**  (SCI)  .  When  added,  weights 
are  often  assigned  to  each  index,  such  that  the  sum  of  the 
weights  add  to  one  (W  ^  w  ^  1}  • 

sex  a  8PX  X  CPI  (6) 

freighted  Index  a  (SPX)  *  (CPX)  (7) 


IS 


The  different  variations  of  these  individual  methods  and 
combinations  present  a  multitude  of  choices  for  calculating 
a  past  performance  factor. 


Holeman  Model  fl975;3K 

One  of  the  earliest  EAC  models  studies  was  done  by  J.B. 
Holeman  (1975) .  Holeman  categorized  cost  estimation  as 
either  parametric  (based  on  cost  estimating  relationships 
(CER)),  or  engineering  (“bottom  up“) .  Holeman  then 
proposed  another  approach  category.  This  “EAC  approach'* 
would  incorporate  data  from  the  contract  and  managerial 
judgment  which  he  thought  would  produce  a  better  estimate  of 
the  final  contract  cost  (2 7; 2-^3}.  Three  methods  were 
presented . 

The  first  method  used  the  BAC  of  the  contract  and 
multiplied  it  by  the  cumulative  cost  performance  index 
(CPI,)  (27ao).  (Since  CPI  ?  •  I  /  CPI^,  dividing  the  BAC 
by  the  CPI  ,  is  an  equivalent  method.) 

EAC  s  BAC  X  CPX  ,  (3) 

EAC  m  SAC  /  CPX  .  (9) 

Equations  3  and  9  will  give  the  same  EAC. 

The  second  method  used  judgmental  evaluation  of  contract 
changes,  inflation,  schedule  variations,  overhead  changes 
and  unexpected  technical  problems  (27:17-21).  Of  these  five 
factors,  contract  changes  contributes  “almost  half  of  the 
average  weapon  system  cost  growth"  according  to  a  Government 
Accounting  Office  Report  in  1973  (27:16).  The  formula 
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requires  subjective  estimates  c£  four  of  the  factors  be 
determined  by  qualified  ^..^dividuals  ?nd  assumes  that 
technical  problems  are  reflected  in  the  CPI  p. 

EAC=  ACWP+[(BAC-BCWP)  X  FPF]-f CONTRACT  CHANGES 

••■SCHEDULE  VARIATIONS  (10) 

Where 

PPF  =  CPI p  +  Inflation  (decimal) 

+  Overhead  (decimal) 

CPI p  =  Unexpected  Technical  Problems 

(27:22) 

Example: 

C':  itract  changes  =  $.8M 
Inflation  change  =»  -04 
Overhead  increase  =  .30 
CPI  ,  =»  1.15 

Schedule  variations  =  $.6M 

EAC  =  ACWP  +  (BAC-BCWP)/(1.15t.04+.3)  +  .8M  +  . 6M 
Holemans*  third  method  involves  a  set  of  steps  to 
determine  a  range  for  the  performance  factor.  Ranges  for 
the  lower  level  work  element  or  "task  CPI"  are  determined  by 
using  a  three-point,  cumulative  %,  or  a  probability 
distribution  method.  Using  the  three  point  method,  three 
lower  level  tasks  would  have  a  high,  most  likely,  and  worst 
value  for  the  CPI  for  each  task  (27:24).  The  sacond  step  is 
to  do  a  simulation  using  the  CPI  ranges  and  a  Monte  Carlo 
sampling  technique.  The  different  combination  of  CPIs  from 
the  ranges  established  will  proc’uce  a  large  number  of 
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estimates.  The  estimates  are  grouped  into  ranges  with 
relative  and  ciomuiative  frequencies  calculated  and  plotted 
against  the  EAC  dollars.  The  last  step  is  to  analyze  the 
data  and  graph  to  get  an  average  EAC  or  a  range  of  EACs 
(27:24-29).  This  method  provides  flexibility  to  the  EAC. 

Lullar  Model  fl980ilQU 

A  less  flexible  model  presentation  was  done  by  James 
Lollar.  Iiollar  attempted  to  avoid  the  judgmental  problem  of 
determining  the  weights  assigned  to  CPI  and  SPT  (32:5).  In 
his  model  (Equation  11),  the  weights  are  determined  by 
adding  the  absolute  values  of  CV%  and  SV%  and  taking  the 
respective  share  of  the  Total  percent. 

EAC  a  ACWP  +  (BAC-BCWP)  /  PPP  (11) 

Where 

PPF  «  CVf(CPX)  +  SVf( SPT) 

CVf  «  |CV%j  /  (|CV%|  +  jsv%|) 

SVf  =»  lsv%l  /  (lcv%|  |SV%1) 

The  "Converlional  Formula'*  of  Lollars*  project  is  Equation  9 
(32:11-12). 

Example: 

CV  «  -10%  SV  «  2S%  ACWP  «  55  BAC  »  200 

BCWP  50  CPI  «  .91  SPI  «  1.10 

CVf  «  lO/JS  ».28  SVf  «  25/35  =».72 

PPF  »  .3a(.91)  -K  .72(1.1)  =»  1.05 

EAC  =  55  +  (200-60)/1.05  a  197.86 
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Totaro  Model  f 1987; 191 
Totaro  also  proposed  a  model  which  involved  the  weights 
of  indices  as  in  Equation  7.  In  his  model,  the  weights 
change  as  the  contract  progresses  and  are  a  function  of 
percent  complete  (45:29-30) . 

SAC=  (BAC-BCWP)  /[.2S-.25{BCWP/BAC)]SPI 

+  [.7S+.2S(BCWP/BAC>]CPI  +  ACWP  (12) 

Where 

BCWP  /  BAG  =  %  Complete 
%  SPI  +  %  CPI  =  100% 


(45:31) 

The  initial  percentages  are  determined  by  the  analyst 
depending  upon  program  goals  and  knowledge.  The  ultimate 
goal  of  the  formula  is  that  as  the  contract  approaches 
completion,  SPI  has  a  decreasing  effect  and  CPI  has  an 
increasing  effect  on  the  performance  factor  (45:32). 
Example: 

Using  the  same  data  as  in  the  Loliar  example. 

EAC  «  200-5Q/PPP  +  55 

PPF“»(.  25-.  25  ( 50/200)  ]  1.1  [  .75-4*. 25(50/200)  ]  .91 

EAC  =•  150/. 945  +  55  «  213.73 


Blvthe  1982..  Study.  ( 1982M4)  . 

A  major  problem  with  the  models  proposed  by  Holeman, 

Lo*.' ,  and  Toturo  is  that  they  were  not  evaluated  against 
actual  programs.  In  1982  Blythe  compared  three  models  using 
actual  program  data.  These  models  were  the  Loliar  Model 
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(Equation  11),  Parker  Model,  and  the  '•Conventional  Formula" 
(Equation  9)  -  as  well  as  the  contractors'  EAC  (6; 6-7).  The 
Parker  Model  was  developed  in  1980  and  has  fixed  weights  for 
SPI  and  CPI  but  the  model  by  Parker  was  not  found  during  the 
research  period  (13:3). 

Parker  Model: 

BAC  =  ACWP+  (BAC-BCWP)  /{.3  SPI  +.7CPI)  (13) 

(6:4,9) 

Blythe  evaluated  the  models  using  data  from  26  programs 
(7  RSD,  19  Production)  and  compared  the  results  at  six 
percent  completion  points.  In  order  to  standardize  the  data 
with  respect  to  contract  changes,  the  final  report  BAC  was 
used  for  each  completion  point.  Any  difference  between  the 
final  BAC  and  the  actual  BAC  at  each  point  was  added  to  or 
subtracted  from  the  calculated  EAC  at  that  completion  point. 
The  actual  BAC  was  used  to  calculate  the  percent  complete 
(6:4-5) . 

The  contractor's  EAC  and  the  Lollar  Model  were  compared 
first.  The  Lollar  Model  was  closer  to  the  final  BAC  only 
37%  of  the  time.  The  Parker  Model  was  then  run  using 
different  combinations  of  weightings  (.1  increments).  Based 
on  the  lowest  coefficient  of  variation  (  Std.  Deviation  / 
mean) ,  the  study  concludes  that  weightings  of  20%  SPI  and 
80%  CPI  provided  the  best  estimates  among  the  combinations. 
All  models  and  weighting  combinations  were  then  compared 
with  the  0%  SPI  and  100%  CPI  representing  the  "Conventional 
Formula".  The  method  for  selecting  the  most  accurate  model 
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is  not  clear  but  appears  to  be  an  evaluation  including  the 
smallest  standard  deviation,  mean  closest  to  the  BAG,  and 
the  range  of  the  estimates  (6:6-8). 

The  conclusions  of  the  study  were  that  the  Lollar  Model 
was  the  least  accurate  with  the  contractor  EAC  being  the 
most  accurate.  The  closer  to  contract  completion  (95% 
completion  point)  all  models  produced  accurate  estimates 
(6:7).  The  contractor  EAC  was  consistently  under  estimating 
and  showed  a  small  standard  deviation  which  interested 
Blythes'  associates  to  expand  on  these  characteristics. 

Blvthe  1984  Study  (1984:15). 

Based  on  the  above  characteristics  of  the  contractor's 
EAC,  Blythe  extended  his  research  in  1984  to  develop  an 
adjustment  factor  based  on  the  contractor's  EAC  (5:2).  The 
factors  were  developed  by  regression  analysis.  Parts  of  his 
report  have  been  lost  (Appendix  A  showing  details  of 
formula)  but  the  formula  produced  is  shown  below; 

ABAC  =  SAC  /[. 91084*. 0892  (BCWP  /  BAG)]  (14) 

(5:3) 

The  AEAC  is  the  "adjusted  contractor  EAC". 

Blythe  also  developed  a  formula  to  calculate  the  standard 
error  (SE)  of  the  contractor's  EAC; 

SE  a  EAC  X  [. 1289-. 0925 (BCWP/BAC) ]  (15) 

The  SE  could  then  be  used  to  calculate  a  confidence  interval 
for  the  adjusted  EAC: 
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AEAC  ±  Z(SE) 


(16) 


(5:3) 

Z  represents  the  value  from  a  normal  distribution  table. 

Example : 

BCWP  =  55  BAG  =  250  Contractor's  EAC  =  275 
ABAC  =  275/[.9108  +  .0892(55/250)]  =  295.57 
SE  «  275  [ .1289-. 0925(55/250) ]  =  29.85 

Crver  and  Balthazar  Study  (1986; 17 K 

Cryer  and  Balthazar  looked  at  the  database  used  by  Blythe 
and  hypothesized  that  evaluating  the  database  by  R&D  and 
production  contracts  would  validate  different  models  for 
each  phase  (13:1).  Other  than  stratifying  the  database,  the 
same  models  and  methods  were  used  (13:2-4).  The  standard 
error  of  the  estimate,  standard  deviation,  and  coefficient 
of  variation  were  used  to  evaluate  the  models'  predictive 
accuracy  (13:4-5).  Results  agreed  with  those  of  Blythe  in 
that  the  contractor's  EAC  was  the  best  overall  predictor. 

The  adjustment  factor  models  that  Blythe  calculated  by 
regression  analysis  were  different  in  the  Balthazar  and 
Cryer  study. 

However,  Balthazar  and  Cryer  findings  on  the  best  weights 
were  different  from  Blythes'.  Based  on  the  coefficient  of 
variation,  the  .1  SPI  and  .9  CPI  had  the  lowest  coefficient 
of  variation  for  the  combined  and  production  but  not  for 
R&O.  Weights  of  .2  SPI  and  .8  CPI  were  second  with  the  best 
R&D  coefficient  of  variation.  The  .3  SPI  and  .7  CPI  weights 
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were  the  next  best  combination.  All  three  combinations  were 
within  1%  of  each  other.  If  standard  error  cf  the  estimate 
was  used  for  selection,  the  .4  SPI  and  .6  CPI  combination 
was  the  lowest  for  combined  contracts  but  no  significance 
was  stated  by  the  authors  (13:5). 

Wallender  Study  C1986tl8). 

In  1986  Wallender  wrote  a  paper  justifying  what  was 
called  the  "Hq  AFSC  EAC  Formula”  (46:1). 

EAC  =  ACWP  +  (BAC-BCWP)/(.2  SPI+.S  CPI)  (17) 

(BAG  excludes  MR  if  MR  is  not  expected  to  be  used.) 

(Wallender: 1) 

Blythes ' study  (1982),  an  AFSC  study  led  by  Wallender,  and  an 
Armament  Division  (AD)  study  done  by  Rutledge  and  Dinato 
were  given  as  evidence  for  the  weights  (46:2-3). 

The  Hq  AFSC  Study  compared  44  contracts  using  EAC 
calculations  based  on  the  20%  and  80%  weighting.  These 
contracts  were  from  the  Ballistic  Missile  Office  (BMO)  which 
was  considered  as  having  the  best  EAC  analysis  techniques 
based  upon  a  May-June  1985  Program  Financial  Review  (PFR) 
conducted  by  Hq  AFSC  (46:2).  Of  the  contracts  compared,  all 
but  five  were  within  10%  of  the  BMO  estimates  using  the  AFSC 
formula  (46:2). 

The  AD  study  used  a  database  of  two  Production  and  13  R&D 
contracts  to  evaluate  the  AFSC  Model  (Equation  17)  and  a 
model  using  SCI  which  was  called  the  "OSD  model”  (46:3). 

EAC  =  ACWP  +  (BAC-BCWP)/  (CPI  X  SPI)  (18) 
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(46:3) 


A  percent  variance  was.  calculated  which  consisted  of  the 
difference  between  the  calculated  EAC  and  final  ACWP  +  MR 
divided  by  the  final  ACWP  +  MR  times  100%. 

Example: 

EAC  =  100 

ACWP  =  110 

MR  =  5 

%  Variance  =  [(100  -  115)  /  115]  100%  =  -13% 

The  variance  was  calculated  at  25,  50,  and  75%  completion 
points.  Results  showed  77%  of  the  contracts  were  within 
±  15%  of  ACWP  +  MR  at  25%  complete,  86%  were  within  ±  10% 
at  50%  complete,  and  93%  were  within  ±  8%  at  75%  complete. 
The  study  showed  the  OSD  Model  EACs  were  10-12%  higher  than 
the  AFSC  formula  (46:3).  The  Wallender  paper  was  the  only 
documentation  that  could  be  found  for  the  AD • and  AFSC 
studies. 

Prise..  (laggf.ie)... 

In  1985  James  Price  did  a  Masters  Thesis  at  the  Air  Force 
Institute  of  Technology  comparing  EAC  formulas  found  in  the 
Cost  Performance  Report  Analysis  (CPRA)  program  used  by 
financial  analysts  in  the  Air  Force  Wright  Aeronautical 
Laboratories  (AFWAL)  at  Wright-Patterson  AFB.  The  CPRA 
program  had  six  formulas  but  Price  excluded  the  formula 
using  trend  extension  based  on  the  regression  of  ACWP.  The 
five  formulas  are  presented  below: 
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BAG  =  ACWP  +  (BAG  -  BGWP)  /GPI,  cur  (19) 

BAG  a  ACWP  +  (BAG  -  BGWP)  /CPI  , c„  (20) 

BAG  =  ACWP  +  (BAG  -  BGWP)  /CPI  3  .(See  Appendix  B)  (21) 

BAG  s  ACWP  +  BTC  (22) 

Where 

ETC  =  [100-(Cost  Var.  %)+.75(Schdule  Var.  %) ]  x 

BCWR  /  100 

BAG  s  BACC  ■¥  BTCS  (23) 

Where 

EACC  =  (.12  X  Eq.18)  +  (.64  X  Eq.l9)  +  (.24  X  Eq.20) 

ETCS  «  (months  behind  Schedule)  x  ACWP  Rate  x  .75 
ACWP  Rate  =  ACWP/Total  contract  completed  months 

(40:10-12) 

Price  used  57  on-going  R&D  programs  as  his  database 
(Price: 15).  The  models  were  evaluated  by  regressing  ACWP 
against  the  calculated  EAC.  The  highest  coefficient  of 
determination  (R  ^  )  was  used  to  determine  the  best 
predictor  (40:27).  Results  showed  Equation  21  to  be  the 
best  predictor  (40:31). 

Reidel  and  Chance  Study  (1989:20). 

A  follow-on  study  to  the  Price  research  was  done  by 
Chance  and  Reidel  (Reidel: 2) .  This  study  was  a  comparative 
study  of  the  EAC  formulas  (Equatio.ns  19,  20,  21)  in  Prices' 
thesis,  two  equations  (Equations  9,  18)  taught  in  the 
Defense  Systems  Management  College  (DSMC)  Contractor 
P6rformanc_e_Mea3urement  course,  and  two  weighted  formulas 
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(Equations  17,  24)  (41:4-6).  Algebraically,  Equations  9  and 
20  are  the  same  equation. 

Equation  9: 

EAC  =  BAC  X  ACWP  cua,  /  BCWP  cu* 

Ec[uation  20: 

EAC  =  ACWP  +  (BAC-BCWPcua,  )  X  (ACWP  /  BCWP  c^) 

EAC  =•  BAC  X  ACWPc^  /  BCWPc«„ 

EAC  a  ACWP  +  (BAC  -  BCWP)  /  [(X)CPI  +  (l-X)SPI]  (24) 

Where 

X  =  %  complete 

(41:5-6) 

This  study  used  16  R&D  contracts  and  40  Production 
contracts  consisting  of  aircraft,  avionics,  and  engine 
(41:29-70) .  The  evaluation  looked  at  different  percent 
completion  points  (25%, 50%, 75%, 100%)  and  compared  the  final 
ACWP  against  the  calculated  EAC.  The  methodology  used  for 
picking  the  "best"  predictor  was  the  lowest  Average  Absolute 
percent  deviation  with  a  crosscheck  using  Average  Rank  order 
at  the  specific  percent  completion  points  and  for  overall 
program  (41:18-20), 

Results  of  the  study  are  shown  in  Table  2  (41:29-70). 

The  parenthesis  indicates  the  number  of  programs  used  in  the 
calculations.  The  numbers  represent  the  equations,  as 
numbered  in  this  thesis,  and  represents  the  best  predicting 
formula  for  that  completion  stage.  Since  Equation  9  and  20 
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TABLE 2 


SUMMARY  OF  REIDEL  AND  CHANCE  STUDY 


R&D 

Percent  Complete 

25% 

50% 

75% 

100% 

Overall 

18 

21 

21 

17 

18,20 

18,24 

21 

21 

17,18,20 

18,21 

19 

18 

21 

21 

18,21 

Production 

ESSSll^fll 

18 

21 

18 

18,17 

17 

18 

17 

18 

17,20 

24 

20 

18,24 

24 

17,20 

Note:  The  numbers  listed  In  each  cell  refer  to  the  EAC 
equation  number  assigned  in  this  research.  The  following 
list  details  the  index  (Past  Performance  Factor)  used 
in  each  formula: 

Equation  Index 

17  (.2)Cumulative  SPIe  4-  (.8)Cumulaiivc  CPIc 

18  Cumulative  CPIe  x  Cumulative  SPIe 

19  Monthly  CPIe 

20  Cumulative  CPIc 

21  3  Month  Average  CPIe  (Sec  Appendix  B) 

24  (X)Cumulaiivc  CPIe  x  ( l-X)Cumulaiivc  SPIe 


give  the  same  results  only  Equation  20  will  be  listed  in 
Table  2. 

Covach  (NAVSEA)  Study  f 1981: 11). 

The  two  DSMC  formulas  were  originally  obtained  from  a 
Navy  sponsored  contract  to  ManTech  International  Corporation 
titled  A  Study  to  Determine  Indicators  and  Methods  to 
Compute  _Estimate_at_  completion  (12:1).  This  study  was  the 
basis  for  the  Defense  Systems  Management  College  (DSMC) 
Modular  6  of  the  Contractor  Performance  Measurement  CQurse 
(14:6).  The  study  evaluated  performance  factor  formulas, 
regression  models,  and  the  use  of  manpower  as  an  estimator 
for  forecasting  EACs  (12:1-2). 

The  evaluation  criteria  for  the  ManTech  study  were  based 
on  assessing  three  basic  principles. 

1.  Accuracy  -  A  method's  estimate  of  costs  at 
completion  (EAC)  should  generally  be  equal  or  close 

to  the  contractor's  actual  cost  at  completion. 

2.  Timeliness  -  A  method  should  be  capable  of 
producing  a  reliable  EAC  as  early  as  possible  in 

the  life  of  the  contract. 

3.  stability  -  A  method  should  not  produce  EACs  which, 
on  a  month  to  month  basis,  vary  widely. 

(12:21) 

Three  methods  were  used  to  evaluate  12  formulas  using  a 
database  of  six  Navy  programs. 


BAC  a  ACKP  ♦  CPl^a, 

(SCVR) 

(25) 

SAC  a  AOrP  -f  CPXn 

(SCVS) 

(26) 

SAC  a  ACVP  4'  CPXk 

(BCVS) 

(27) 

SAC  a  ACVP  CPZ,j2 

(BCVS) 

(29) 
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EAC  s  ACVP  *  CPIp  (BCWR) 

(29) 

SAC  =  ACWP  +  (CPIp  /  SPDBCWE 

(30) 

EAC  5  CPXp3  (BAC) 

(31) 

EAC  3  BAC/SPI 

(32) 

EAC  3~CPI  p3  (BAC) 

(33) 

EAC  3 p,  (BCWR)  4-  ACWP 

(34) 

BAC  (BAC) 

(35) 

BAC  (BCWR)  •¥  ACWP 

(36) 

(12:23-24) 

The  evaluation  used  three  methods:  10%,  BAC,  and  liRE. 

The  "10%  method"  compared  the  calculated  EAC  to  the  final 
ACWP  to  see  if  it  is  within  t  10%  of  the  ACWP.  If  the 
calculated  EAC  is  within  this  range  more  than  7S%  of  the 
time,  the  formula  was  classified  as  a  success.  Meeting  the 
criteria  less  than  S0%  of  the  time  was  classified  as  a 
failure  and  between  75%  and  S0%  was  rated  indifferent.  The 
"BAG  method"  looked  at  where  the  EAC  fall  between  the  BAC 
and  ACWP.  If  the  EAC  fell  on  the  high  side  of  the  BAC  or 
ACWP  then  the  distance  should  be  less  than  the  difference 
between  BAC  and  ACWP  for  a  success.  The  "LR£"  method 
compared  the  contractor's  EAC  and  the  calculated  EAC  to  the 
ACWP.  If  the  EAC  calculated  was  better  than  the  contractors 
EAC  it  was  successful.  This  was  done  on  a  monthly  basis. 

The  life  of  a  contract  was  then  broken  into  quintals  (20% 
increments)  and  the  formulas  evaluated  by  phases  (12:34- 
27) . 
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A  scoring  system  was  established  giving  a  numerical  score 
of  1  for  successes^  0  for  indifference#  and  -1  for  failures. 
Three  phases  of  contract  completion  were  defined  with  the 
early  phase  being  the  first  2  quintals  (20%) ,  the  middle 
being  the  middle  3  quintals,  and  the  late  phase  being  the 
last  two  quintals.  The  formulas  were  evaluated  by  scoring 
them  by  the  three  methods  cibove  and  by  phases  (12:28-31). 
Results  by  stage  are  listed  below: 

Early  Middle  Late 

Eq.  29,30,34  Eq.  27,29,30,34  Eq.  28,34,36 

(12:62) 

The  regression  evaluation  used  three  regression 
approaches  against  four  different  curves  (linear, power, 
exponential,  log)  plus  the  **Nttw  SAMSO**  model  which  will  be 
discussed  later  (12:31-35).  The  same  evaluation  methods  and 
database  were  used.  Results  showed  three  approaches  and 
curves  were  best  relative  to  the  regression  equations  only. 
The  first  method  was  regressing  ACWP  against  BCWP  and 
computing  ACWP  where  BCWP  «  BAC  with  a  linear  curve.  The 
second  method  was  regressing  the  CPI  as  a  function  of  time 
and  matching  the  final  CPI  to  the  BAC  .  The  curve  equation 
was  a  power  function  equation.  The  last  method  was  to 
regress  ACWP  and  BCWP  to  time  and  find  the  final  time  where 
BCWP  »  BAC  and  using  this  time  in  the  ACWP  regression 
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equation  to  calculate  an  EAC.  This  curve  used  was  an 
exponential  equation  (12:30-35). 

Additional  programs  (9  R&D,  2  Prod.)  were  added  to  the 
database  in  order  to  do  an  analysis  of  manpower  as  an 
indicator  for  calculating  EACs.  The  study  also  looked  at 
developing  EACs  from  lower  level  Work  Breakdown  Structure 
tasks  (Level  2) .  Nothing  was  fcund  to  indicate  manpower 
would  be  a  good  indicator  to  use  for  EAC  formulas. 
Investigation  of  developing  estimates  at  lower  level  tasks 
did  not  provide  a  better  EAC  (12:20). 

A  final  aspect  of  this  study  involved  interviews  with 
leaders  in  the  C/S  community.  Results  revealed  the 
following  opinions: 

1.  Trends  rarely  reverse  themselves, 

2.  Managers  want  to  use  simple  forecasting  methods. 

3.  Manpower  loading  data  were  under  utilized  by 
analysts. 

4.  Initial  trouble  or  problems  appear  in  the  schedule 
area. 

(12:7) 

An  update  to  the  NAVSEA  study  was  published  in  1932 
(24:1).  The  extended  research  increased  the  database  to  21 
programs  and  evaluated  a  different  EAC  method  called  the 
"Range  Method" .  Results  of  this  study  did  not  change  the 
rankings  of  the  formulas  in  the  earlier  stv’.dy  or  provide  a 
better  method.  Further  work  with  regression  analysis  was 
done  by  adding  three  hyperbolic  curve  equations  and  using 
the  same  regression  approaches. 


31 


Y  =  a  +  to  /  X 

(37) 

3  1  /(a  +  b  X) 

(38) 

/Y  3  a  +  b  /  X 

(39) 

(24:23) 

The  update  produced  two  other  regression  approaches  that 
should  be  considered.  The  two  approaches  used  Equation  37 
and  the  second  and  third  methods  stated  above  in  the 
regression  review  for  this  study  (24:24). 

Land  and  Preston  Study  f 1980: 9^ . 

Some  of  the  Services  have  developed  computer  programs 
that  do  CPR  and  C/SSR  analysis.  These  programs  usually  have 
many  EAC  formulas  from  which  the  analyst  can  pick.  One  of 
these  programs  is  the  Automated  Financial  Analysis  Program 
at  ESD.  The  program  allows  the  analyst  to  pick  three  EAC 
formulas  that  will  give  him/her  a  range  of  values  for  an  EAC 
(1:64-65) . 

General  equation  is  FAC  =•  ACWP  +  ETC 

ETC  =  (1  -  CVI  Car  )aCTO  (40) 

Where 

CVI  «  CV  /  BCWP 

ETC  3  [1  -  (CVX.  +  CVX^i  -f  CVI  ^2/3)]BCWR  (41) 

Where 

m  “  current  monthly  data 

ETC  3  Performance  Paotor  determined  by  analyst 

ETC  3  (1  -  CVDBCWR  (42) 

ETC  3  (X  -  (BCWP,  -  ACWP,  /  BCWP,))  BCWR  (43) 
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Where 


BCWP3  =  BCWPcur  +  BCWPcu^-l  +  BCWP  cur-2 
ACWP3  =  ACWPcur  +  ACWPc„-i  +  ACWP  cur-2 

ETC  =  [1  -  CVWCCVI)  +  SVW(SVI)]BCWR  (44) 

Where 

CVW  »  Cost  Variance  Weight 
SVW  =  Schedule  Variance  Weight 

(30:23-27,  l:A2-5-6) 

This  ESD  program  (linear  models)  and  the  Karsch  Model  (a 
nonlinear  model  reviewed  in  the  regression  section)  were 
compared  in  a  Masters  Thesis  by  Captain  Land  and  Captain 
Preston  (30:1).  The  methodology  used  for  the  accuracy  of 
prediction  was  the  absolute  value  of  (EAC-ACWP)/ACWP  or  the 
error  %  (30:32).  The  research  used  20  programs  which  were  a 
mix  of  R&D  and  Production  contracts.  Five  more  contracts 
were  added  in  order  to  calculate  an  average  value  for  b  j 
which  is  part  of  the  Karsch  model.  The  research  showed  the 
following  results: 

1.  A  non  linear  model  was  no  more  accurate  than  the 

linear  model. 

2.  At  different  percent  completion  points  the  non 
linear  model  was  no  more  accurate  than  the  linear 

model . 

3.  The  b  2  value  for  aircraft  should  be  1.033  and  the 

b  2  number  is  very  critical  to  the  accuracy  of  the 

£AC  calculation. 

(30:40-49) 
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Bright  and  Howard  Study  (1981: 12 K 

Another  computerized  program  is  the  Automated  Contractor 


Performance  Measurement  System  (ACPMS)  used  by  the  Army 
Missile  Command  (MICOM)  (7:2).  Bright  and  Howard  looked  at 
the  ACPMS  formulas  and  compared  them  to  an  "improved  method” 
of  developing  a  performemce  factor.  This  method  developed  a 
performance  factor  by  multiplying  the  CPI  by  the  SPI  (7:17). 

BAG  =  ACWP  +  (BAC-BCWP)/  (CPI  X  SPI)  (45) 

(7:17) 

The  formulas  used  in  the  ACPMS  are  as  follows: 

General  equation  EAC  =  ACWP  +  (BAC  -  BCWP)P 


P  s  1  /  CPI, 

(46) 

Pal/  CPI  9 

(47) 

PS  1  /  ^  la 

(48) 

P  a  I  /  CPI 

P  a  Detaraiaed  by  tha  analyst 

(49) 

The  following  formulas  weight  the  indices  and  have 

general  formula  as  above  except  for; 

P  «  1  /  A 

the  same 

A  a  [W  i(SPI)  ♦  W  a(CPl>  ]  /  (W  ^  ♦  W  ^  ) 

Where 

W  1  W  2  a  1 

Different  weight  combination: 

(50) 

WjS.S  WjS.S 

(51) 

Wjs,7S  Wjs.as 

(52) 

W  1  ai.o  Wj  ao.o  same  as  l  /  SPI 
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(53) 

Two  regression  approaches  (See  Appendix  C  for  Approaches) . 

(7:4-7) 

The  database  consisted  of  11  Army  R&D  contracts  which 
were  evaluated  at  different  per  cent  completion  points. 

Data  were  adjusted  for  rebaselining,  additional  scope,  and 
contracts  not  completed  (7:10-12).  This  normalized  the  ACWP 
and  allowed  the  monthly  calculation  of  EAC  to  be  compared  to 
it.  The  forecasts  were  scored  by  month  for  the  best 
forecast  (1  point)  and  fractions  of  a  point  were  given  for 
estimates  closest  to  the  best  EAC  for  that  month  (7:14-15). 

A  separate  analysis  of  different  CPI  x  SPI  combinations 
resulted  in  a  six  month  moving  average  for  CPI  and  a 
cumulative  SPI  being  the  best  predictors  of  final  cost 
(7:17).  This  equation  was  then  added  to  the  other  ACPMS 
formulas  for  comparison. 

The  results  of  the  study  showed  the  SPI  formulas, 
regression  models,  and  CPI  x  SPI  did  better  in  the  first  30% 
of  the  contract.  After  30%  the  cumulative  CPI  and  moving 
average  CPI  did  better  at  predicting.  The  CPI  x  SPI 
combination  did  excellent  from  31%  to  80%  complete  and  was 
rated  good  after  80%  to  contract  completion  (7:15-19). 

Other  sources  of  EAC  formulas  found  during  this  research 
were  the  System  361/362/363  courses  taught  at  The  Air  Force 
Institute  of  Technology  (AFIT)  and  Performance  Analyzer 
computer  program. 

The  Performance  Analyzer  was  developed  to  analyze  CPR 
and  C/SSR  reports  (42 :i).  The  EAC  section  contains  seven 
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different  EAC  formulas  ranging  from  CPI  moving  averages  to 
regression  models  (42:85-88).  The  formulas  do  not  include 
MR  but  the  program  allows  the  analyst  to  add  MR  after  the 
EAC  is  calculated  (42:40).  The  Air  Force  Institute  of 
Technology  provides  a  list  of  EAC  formulas  to  students 
attending  during  C/S  courses.  These  equations  are  not 
developed  by  AFIT  but  gleaned  from  other  studies.  But  the 
nximber  of  analysts  that  go  through  the  courses  warrants 
mention  of  the  formulas  presented  there.  Appendix  c  shows 
the  formulas  for  the  computer  program  and  AFIT  courses. 

This  has  been  a  literature  review  of  models,  studies,  and 
sources  of  EAC  formulas  that  use  Past  Performance  factors  to 
forecast  final  costs.  Different  formulas  were  found  to  be 
better  predictors  at  certain  percent  completion  points. 
Different  weighting  combinations  were  also  tested  to  find 
the  best  predicting  combination.  The  next  category  listed 
in  Table  1  is  Regression  Analysis. 

Regression  Analysis 

Regression  analysis  looks  at  the  statistical  relationship 
between  two  variables.  The  variables  consist  of  a  dependent 
variable  (Y)  and  an  independent  variable  (X)  (37:23-26). 

The  relationship  between  the  dependent  variable  and  the 
independent  variable  can  exhibit  a  linear  relationship 
(straight  line)  of  nonlinear  relationship  (curve) .  Neter 
explains  that  there  are  three  types  of  regression  models  - 
linear,  intrinsically  linear,  and  nonlinear.  The  linear 
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models  have  a  linear  relationship  with  respect  to  the 
parameters  (coefficients  and  y- intercept) .  Intrinsically 
linear  implies  the  parameters  are  nonlinear  .’out  can  be 
transformed  by  mathematical  manipulation  (logs,  square, 
inverse,  square  root)  into  linear  parameters.  Nonlinear 
model  parameters  can  not  be  transformed  (37:142-145,  550- 
551) .  A  regression  equation  can  have  one  independent 
variable  or  multiple  independent  variables  to  explain  the 
relationship. 

Single  Variable  Regression  Equation  (Linear) : 

Y,sB„  +  BiXi  +  Ei  (54) 

Where 

Y i  -  Calculated  value 

B  a  -  Y  intercept 

B I  -  Slope  of  line 

X i  -  Value  of  X  . 

E  j.  -  Random  Error 

(37:31-34) 

Multiple  Variable  (Linear) 

YiSBa+i^iiCi+BjXj+Bi  (55) 

(37:31-34) 

One  of  the  assumptions  of  linear  regression  analysis  is 
that  the  expected  value  of  the  random  error  term  is  zero 
(37:32) .  With  this  assumption  the  random  error  term  will 
drop  out  of  the  above  formulas.  Computer  analysis  is 
required  for  fitting  the  line  and  doing  the  calculations  for 
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the  necessary  statistics  to  analyze  the  results  (standard 
error, R  F~test,  t-test) . 

Karsch  Study  f 1974 : 2  K 

Computer  analysis  programs  for  Cost/Schedule  reports  such 
as  CPRA  (Price  Thesis) ,  ACPMS  (Army) ,  and  Performance 
Analyzer  include  linear  and  nonlinear  regression  models 
(40:11,  7:7,  42:88).  One  of  the  regression  models  was 
developed  by  Karsch  in  1974. 

Karsch  Model: 

y  a  b  1  X  b  ^  (56) 

(28:13) 

One  of  the  reasons  behind  this  formula  was  long  term  growth 
of  ACWP  and  BCWP  was  shown  to  be  nonlinear.  The 
nomenclature  b  ^  and  b  2  are  non-dimensional  growth  factors 
(28:13-14).  This  model  also  has  a  constrained  mode  where  b 
2  is  held  constant  and  an  unconstrained  mode  where  it  is 
calculated  each  month  (28:16). 

A  comparative  study  was  done  with  one  sample  using 
percent  cum  variance,  constrained,  and  unconstrained  models. 
The  selection  criteria  was  the  method  that  generated  an  EAC 
closest  to  the  final  ACWP  the  earliest  (28:18).  The  results 
showed  that  the  constrained  model  was  closer  to  the  final 
ACWP  up  to  the  93%  completion  point.  All  three  models  were 
close  at  the  98%  completion  point  with  percent  cum  cost 
variance  better  after  98%  completion  (28:23). 
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since  the  constrained  model  uses  b  2  as  a  constant 
value,  Karsch  conducted  another  study  in  1976  to  determine  a 
value  range  for  b  2  for  production  contracts  (29:1).  A 
database  of  13  production  aircraft  and  missile  contracts  was 
used  to  compare  the  same  formulas  as  in  Karschs'  1974  study 
(29:3).  The  results  for  aircraft  data  showed  that  up  to  38% 
completion  the  constrained  model  (b  2  =  1.053)  was  better. 
From  88%  to  98%  contract  completion  both  Karsch  models  were 
better  predictors  and  from  98%  the  percent  cum  variance 
formula  was  better.  When  comparing  the  constrained  to  the 
contractor  EAC  the  contractor  EAC  was  better  after  95% 
completion  (29:5-9) . 

The  missile  data  showed  the  constrained  model  (b  2  = 

1.041)  a  better  predictor  up  to  75%  completion  with  %  cum 
variance  better  after  this  point.  When  the  contractor  EAC 
and  the  constrained  model  were  compared,  the  contractor  EAC 
was  better  between  0-20%  and  88-100%  (29:11-15). 

When  evaluating  the  value  for  b  2/  the  range  for  the 
production  contracts  was  from  ,94-1.13  with  the  most  common 
values  between  1. 0-1.1.  The  range  did  not  differ 
significantly  between  aircraft  and  missile  contracts 
(29:16) . 

He_vdiP_ge,3^(.S.AIlS_0_L_St.udY  L1977 : 7 ) . 

The  Space  and  Missile  Systems  Organization  (SAMSO) 
produced  a  model  that  also  used  a  regression  technique 
(26:8).  The  current  month  BCWP  and  the  previous  five  months 
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BCWP  were  regressed  against  time  by  Least  Squares 
regression.  The  same  procedure  is  done  for  the  ACWP.  The 
point  where  BCWP  equals  BAC  is  the  point  where  the  ACWP 
regression  line  is  read.  This  ACWP  value  is  the  EAC  (26:8- 
9) .  This  method  along  with  the  Xarsch  model  (Equation  56 
constrained)  and  ESD  automated  program  Equations  40,  41,  42, 
43  were  compared  against  an  unknown  number  of  data  points 
(26:11-20).  The  results  showed  the  Karsch  model  (Equation 
56)  gave  better  predictions  from  the  third  to  the  seventh 
month  and  27th  to  42th  month  with  the  ESD  model  Equation  41 
providing  better  results  between  months  8-26  (26:18). 

This  study  also  proposed  a  new  SAMSO  model.  Curves  were 
fitted  to  the  expenditure  data  with  the  best  fit  resulting 
from  a  modified  Erlang  equation. 

Y  s  a  X  “  e  “  (57) 

Where 

X  =  number  of  cumulative  months 

Y*  ACWP, BCWP  or  BCWS 

The  ACWP, BCWP  and  BCWS  are  regressed  with  this  equation  and 
the  same  basic  SAMSO  procedures  are  followed  except  now 
schedule  can  be  projected  (BCWS  =»  BAC)  (26:22-23). 

The  original  SAMSO  procedures  are  followed  with  the  BCWP 
regressed  by  the  Erlang  equation.  The  BAC  time  (T  b^c  )  is 
where  the  BCWP  is  equal  to  the  BAC.  The  time  is  ACWP 
regression  equation  and  the  value  at  T  bac  is  the  EAC. 
Regressing  BCWS  to  the  BAC  line  will  give  a  schedule 
variance  in  time  (26:21-21A). 
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Using  the  constrained  "New  SAMSO"  model  against  the  above 
equations  the  results  were  interpreted  by  the  author  that 
the  New  SAMSO  was  a  better  model  for  predicting  the  EAC 
(26:28)  . 

The  regression  type  of  models  require  the  use  of  a 
computer  as  compared  to  a  simple  calculator  with  the  Past 
Performance  Factor  models.  Regression  and  performance 
indices  are  the  two  major  categories  of  models  but  there 
have  been  other  approaches  tried  in  tb.e  seaxch  for  a  better 
forecasting  model. 

Other  Techniques 

This  category  contains  a  mixture  of  models  and  approaches 
which  are  seldom  used.  One  of  the  areas  has  been  in  the  use 
of  Bayesian  probability  to  predict  EACs. 

El-Sabban  Study  (1973 ill. 

The  approach  involves  the  determination  of  probabilities 
of  events  happening  (prior  probability)  and  with  additional 
infozmation  (new  CPR)  these  probabilities  can  be  updated 
(posterior  probability)  (33:725).  The  posterior 
probabilities  are  conditional  probabilities  and  are  centered 
around  Bayes  Theorem. 

P{P/E)  =:  P(Pi  )  P  (E/Pi  )  /E"  P(Pi  )  P(E/P)  (58) 

Where 

F  I  -  n  mutually  exclusive  and  exhaustive  events 

E  -  a  known  outcome 

P(Fi)  ~  prior  probability 
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P(E/F  i)  -  conditional 
occurred 


probeUaility  of  E  given 


(33:726-727) 

In  1973  Zaki  El-Sabban  proposed  the  use  of  Bayesian 
probeibility  to  calculate  an  EAC  (20:1-2).  The  procedure 
derives  a  mean  value  (At)  that  is  the  EAC. 

El-Sabban  Model: 

E(M)  so  M,  Oo  +  Mo  e’  /  (  <To^  +  cV,*  )  (59) 

Where 


Ma  =  ACWP 
Mo  =  bag 
c  =  Mo  /  BCWP 

<Ta  =  .1  /  M. 

(Jo  =>  .05  /  Mo 

(20:6) 

The  variance  could  also  be  calculated  Ui»ing  the  following 
formula . 

V(M)  =  ©2  Co*  a.*  /  (  cTo*  -f  a,*  )  (60) 

Where 

<j  (V)  ^ 

^20:8) 

This  would  provide  an  EAC  and  a  range  which  could  be  updated 
by  each  monthly  CPR. 

The  only  other  study  that  could  be  found  relating  to 
Bayesian  probability  was  done  by  Hayes  in  1977  (25:5),  The 
study  proposed  corrections  to  El-Sabban 's  methodology  in 
that  BCW.<!  should  be  used  for  prior  probability  instead  oi 
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ACWP  and  ACWP  should  be  used  for  updating  posterior 
probabilities  (25:42) . 

This  study  compared  the  Karsch  model (Equation  56), 
Bayesian  approach  with  Hayes  corrections  (Equation  59) ,  and 
the  cumulative  ciirrent  percent  cost  variance  model  (Equation 
9)  used  in  the  Karsch  studies  (25:43-44) .  A  data  base  of 
three  R&D  and  two  Production  programs  were  used  (25:45). 

The  evaluation  was  based  on  how  soon  a  formula  could  predict 
the  final  cost.  The  results  showed  the  Karsch  model  was  a 
better  predictor  three  out  of  five  times  followed  by  the 
Bayesian  method  (25:69). 

'tim  g.9glr.9g-An4iv§la.»„ 

Time-series  analysis  was  another  area  looJced  at  for  EAC 
formulation.  Time-series  analysis  can  be  defined  as  '*a 
collection  of  observations  made  sequentially  in  time" 

^11:4).  Moving  averages,  e-^ponential  smoothing,  and 
regression  analysis  are  different  methods  used  in 
forecasting  in  time-series  analysis  (10:223).  One  of  the 
disadvantages  is  the  method  takes  a  large  amount  of  data 
points  to  work  well. 

There  is  not  much  evidence  of  the  above  procedure  being 
used  extensively.  Sincavage  did  a  study  called  TSARISM; 
Time__Serles_An.tl\Lgis  _for.  Army  internal  .Systems  Management  in 
1974,  Researcih  into  libraries  and  even  contacting 
Sincavage  could  not  produce  a  copy  of  this  study  .  A  paper 
by  Ellsworth  and  Olsen  did  provide  evidence  that  the  B-1  SPO 
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(Special  Program  Office)  in  Los  Angelos  was  using  a  time 
series  analysis  method  as  one  of  their  EAC  techniques 
(38:10).  The  approach  used  a  computer  program  called  GETS A 
to  generate  the  EACs. 

Ft  ~  Sigma  Sfi  *  (1-sigma)  Ft-i  (61) 

Where 

Ft  =  forecast  for  time  t 
Ffi  =  forecast  for  time  t-1 
St-x  =  cost  for  period  t-1 
Sigma  »  smoothing  constant  (O-l) 

(38:11) 

The  Olsen  paper  explained  the  various  methods  and 
computer  programs  used  by  the  SPO  to  come  up  with  an  EAC. 
Regression  analysis,  ••grassroots”  estimates,  trend  extension 
analysis,  and  modified  exponential  smoothing  (Time-series) 
were  used  by  the  SPO  (38:8-13),  The  paper  acknowledged  the 
importance  of  using  more  than  one  method,  the  use  of 
judgment,  and  documentation  in  the  development  of  a  forecast 
(38:23)  . 

ausse  Model  f 1977; Si . 

other  methods  have  also  been  published  such  as  the  Busse 

Model.  This  model  was  based  on  the  sensitivity  of 
ACWP  to  BCWp  (8:22).  This  sensitivity  factor  (e)  is  the 
driving  force  of  the  model. 

r 


SAC  s  Z(BAC) 


(62) 


Where 


e  =  (ACWPc^r  /  ACWP)  /  (BCWPc„  /  BCVJP) 

Z  *  ACWP/(BCWP)  *“ 

(8:26-27) 

Chacko  Study  (1981: 13 K 

Chacko  proposed  an  approach  called  the  "adaptive” 
forecasting  approach  in  which  the  model  is  fitted  to  the 
data  rather  than  the  reverse  (9?  7 5).  Thi.^  approach 
eliminates  the  concern  about  the  linearity  of  the  data 
(9:75).  It  utilizes  a  computer  program  called  MESGRO.  A 
new  equation  is  calculated  for  the  next  month  and  adjusted 
by  the  difference  between  the  forecast  and  actual  cost. 
According  to  the  author, it  takes  about  five  data  points  to 
stabilize  and  produces  accurate  estimates  must  faster  in  the 
short  term  (9:78-81).  There  was  little  detail  in  the  study 
which  was  taken  from  a  periodical.  The  suspicion  is  that 
the  study  was  condensed  because  there  was  no  Bibliography 
included.  No  other  source  of  the  study  could  be  found. 

Weida  Model  (1977:8) . 

Weida  found  that  R&D  expenditures  follow  an  S -shaped 
curve.  The  study  was  focused  on  modifying  the  curve  so  it 
could  be  used  for  Cost/Schedule  forecasting  (47:4).  The  S- 
shaped  curve  is  a  cumulative  form  of  a  bell  shaped  curve 
with  the  inflection  point  being  the  mid-point  of  the  bell 
shaped  curve  (47:5).  The  general  approach  was  that  each 
curve  (break  at  inflection  point)  fits  a  separate  equation. 
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The  one  problem  is  the  R&D  data  have  a  tendency  for 
autocorrelation  which  must  be  eliminated  before  an  equation 
can  be  determined  (47:6).  The  22  programs  showed  that  a 
quadratic  equation  or  logarithmic  equation  fit  the 
curves (47 : 18) . 

The  inflection  point  is  the  last  largest  incremental 
changes  which  is  followed  by  two  periods  of  decreasing 
expenditures.  This  is  the  last  point  of  the  lower  curve  and 
the  first  point  of  the  upper  curve  (47:18).  Actual  cost  are 
used  in  the  equation  to  project  an  EAC  (47:19).  This 
procedure  has  the  flexibility  to  produce  a  worst,  best, and 
most  likely  EACs  and  provide  confidence  intervals  (47:21- 
31) .  The  procedure  takes  the  lower  half  equation  and 
calculates  V  then  substitutes  Y  for  the  Y  in  the  second 
equation  to  determine  the  y-intercept  (a) .  The  y-intercept 
is  placed  in  the  upper  half  equation  so  the  schedule 
variance  can  be  determined  (100%  Schedule  %  variance). 

This  value  replaces  X  and  the  resulting  Y  value  is  the 
percent  the  BAG  is  multiplied  by.  This  study  did  not 
compare  this  formula  against  any  other  formulas.  The 
following  example  will  illustrate  this  procedure. 

Example: 

Lover  half  equation: 

Y  I  a  .5  -  .7  (to  ♦  ,2(t  2  )  *  (63) 

X „  a  46%  complete 

Y  i  =>  .5  .7(.46)  .2(.46)  ^ 

Y  I  a  .86 
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Upper  half  equation; 

Y„  =  .3  +  l.a{X  i)  +  .1(X  2)  ^  (64) 

.86  -  1.8 (.46)  -.1(.46)  ^  =  a  (y- intercept) 
a  =  .011 

schedule  variance  =  1.0  +  .01  =  l.oi 
Y„  =  .011  +  1.8(1.01)  +  .1(1.01)^ 

BAG  =  BAC(Y  a )  (65) 

(47:39-40) 

Summary 

This  chapter  reviewed  the  EAC  literature  and  separated 
the  formulas  found  in  the  literature  into  past  performance, 
regression,  and  other  techniques  categories.  These  EAC 
equations  have  been  compared  for  accuracy  of  forecasting 
related  to  percent  complete,  weighting  percent,  type  of 
contract,  and  type  of  product.  Other  approaches  such  as 
Bayesian  probability,  S-shaped  curve, ’’adaptive”  forecasting, 
and  exponential  equations  (Busse  Model)  were  reviewed.  The 
next  chapter  will  critique  each  study  in  order  to  evaluate 
tlie  merit  of  the  research  conclusions. 
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III.  Analysis  of  Research  Studies 


Chapter  II  reviewed  the  different  EAC  models  and  studies. 
This  chapter  will  analyze  the  studies'  strengths  and 
weaknesses.  The  studies  in  Chapter  III  are  separated  into 
three  categories  according  to  the  type  of  EAC  formula  (Past 
Performance  Factors,  Regression, and  Other).  The  Study  Type 
(ST)  column  of  TeUsle  1  further  divides  the  sources  into: 

1 .  Comparison  Studies 

2 .  Model/Approach 

3.  Follow-on  Studies 

Chapter  III  will  be  separated  according  to  the  categories 
above.  As  stated  in  Chapter  II,  a  "Study"  is  defined  as 
research  that  compares  different  EAC  formulas  using 
Cost/Schedule  performance  data.  "Models"  present  EAC 
formulas  as  single  equations  or  a  sequence  of  steps  or 
techniques  to  produce  an  EAC.  Since  Follow-on  studies  are 
usually  associated  with  Comparison  or  Model/ Approach 
studies,  they  will  be  discussed  with  their  associated  study 
type.  In  order  to  evaluate  the  studies,  evaluation  criteria 
were  established. 

Evaluation  _criter ia 

In  evaluating  a  study  what  should  you  look  for?  C. 

William  Emory,  in  his  book  Business  Research  Methods,  gave 

seven  tests  that  good  research  should  meet. 

1.  The  purpose  of  the  research,  or  the  problem 
involved  should  be  clearly  defined  and  sharply 
delineated  in  terms  as  unambiguous  as  possible. 
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2.  The  research  procedures  used  should  be  described  in 
sufficient  detail  to  permit  another  researcher  to 
repeat  the  research. 

3.  The  procedural  design  of  the  research  should  be 
carefully  planned  to  yield  results  that  are  as 
objective  as  possible. 

4.  The  researcher  should  report,  with  complete 
frankness,  flaws  in  procedural  design  and  estimate 
their  effect  upon  the  findings. 

5.  Analysis  of  the  data  should  be  sufficiently 
adequate  to  reveal  its  significance;  and  the  methods  of 
analysis  used  should  be  appropriate. 

6.  Conclusions  should  be  confined  to  those  justified 
by  the  data  of  the  research  and  limited  to  those  for 
which  the  data  provide  an  adequate  basis. 

7.  Greater  confidence  in  the  research  is  warranted  if 
the  researcher  is  eifperienced,  has  a  good  reputation  in 
research,  and  is  a  person  of  integrity. 


(21:10-11) 

Table  3  summarizes  the  evaluation  of  the  studies  based  on 
general  characteristics  of  the  first  six  tests  in  the  above 
list.  The  seventh  test  will  not  be  considered  since  the 
authors  are  not  known  to  the  researcher  and  theinfluence 
could  bias  the  evaluation  of  the  other  six  tests.  The  six 
tests  are  represented  in  Table  3  as  '^Research  clarity", 
“Documentation",  and  "Methodology".  The  numbers  are  study 
or  model  reference  numbers  from  Table  1. 

"Research  Clarity"  involves  the  structure  of  the  study. 

If  the  purpose  of  the  study  is  clearly  stated  ("Clear 
Purpose")  and  if  the  importance  of  the  study  is  described 
("Importance")  then  a  "Y"  (Yes)  is  reported  in  the  table; 
otherwise  a  "M"  (No)  is  reported.  The  other  Y*s  and 
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ANALYSIS  OF  EAC  MODELS  AND  STUDIES 
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-No  Y  -  Yes  N/A  -  Nol  Applicable  •  -  Minor  problem  @-Found  by  contacting  contractor 


N*s  similarly  report  this  author's  judgment  of  the  reviewed 
studies  relative  to  the  qualitative  criteria  listed.  The 
study  should  also  employ  concise,  simple  wording 
("Understandable") ,  and  define  unusual  terms  or  jargon 
("Terms  Defined") .  Conclusions  should  be  clearly  stated  and 
be  based  on  the  data  in  the  study  ("Conclusions  Stated") . 
Finally,  representing  the  conclusions  as  being  universal 
without  expressing  the  known  flaws  or  limitations  of  the 
study  can  be  misleading.  Thus,  the  limitations  of  the  study 
should  be  described  ("Limitations") . 

The  "Documentation"  section  looks  at  the  written  support 
of  the  research.  At  the  very  least,  the  study  should  report 
the  source  of  the  data  ("Data  Source  Stated") .  Ideally,  it 
should  also  include  the  data  ("Data  Included")  utilized  in 
the  research.  "Calculations"  in  the  research  help  a 
reviewer  to  justify  the  conclusions  of  the  author.  In  any 
type  of  work  (sports,  teacher,  briefings)  preparation  is 
required.  A  "Literature  Review"  of  the  subject  is  the 
preparation  that  makes  research  an  addition  rather  than  a 
repeated  study  effort  that  adds  nothing.  The  last  criterion 
in  this  category  is  "Reproducability".  The  documentation 
should  be  sufficient  to  allow  the  study  to  be  reproduced  by 
another  researcher. 

Analysis  and  collection  of  data  can  be  the  most  difficult 
phase  of  research  (21:11).  The  "Methodology"  section  is 
designed  to  evaluate  the  adequacy  of  the  studies  in  this 
area.  The  World  Book  Dictionary  defines  theory  as  "the 
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principles  of  a  science  or  art  rather  than  its  practice” 
(2:2174).  Under  this  section  the  "Theory  Based  ”  criterion 
is  met  when  the  study  evaluates  or  tests  a  stated  theory 
rather  than  explores  a  database  for  unknown  relationships. 
The  "Sample"  criterion  was  included  mainly  for  the  Model 
studies.  This  indicates  an  example  is  included  for 
illustration  or  a  comparison  of  formulas.  Comparison 
studies  utilize  databases.  The  research  methods  assume 
normalcy  exists  because  of  the  Central  Limit  Theorem 
(23:273).  If  the  Central  Limit  Theorem  assumptions  are  met 
the  sample  size  ("Size"),  as  a  rule  of  thximb,  should  be 
equal  to  or  greater  than  30  (23:275).  The  reliability  of 
the  data  ("Representative")  is  another  criterion.  This 
research  defines  "Representative"  as  data  from  a  C/SCSC 
approved  system  (CPR  data  or  C/SSR  data  with  explanation  of 
system  being  reviewed) .  In  order  to  understand  the  results 
and  the  methods  of  evaluation  a  complete  description  of 
decision  criteria,  models  and  formulas,  and  statistical 
assumptions  must  be  presented  ("Method  Description"). 
"Statistics  Tested"  refers  to  evidence  that  the  assumptions 
were  tested  if  statistical  evaluation  was  performed. 

The  last  section,  "Literature  Source",  describes  the 
availability  of  the  studies,  "OTIC"  refers  to  studies 
located  and  obtained  by  the  Defense  Technical  Information 
Center  (DTIC)  system.  "Periodicals"  refers  to  professional 
journals  or  business  periodicals.  "Service  Libraries"  are 
the  base  or  cost  libraries  located  at  DOD  installations. 
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The  above  criteria  will  answer  the  six  tests  stated  by 
Emory  for  good  research.  The  nomenclature  and  standards  for 
the  evaluation  are  described  below. 

The  analysis  will  look  at  Models/Approaches  and  then 
Comparison  studies.  Computer  programs  and  AFIT  courses  will 
not  be  included.  If  a  criterion  is  met  by  a  study  or  model, 
it  is  assigned  a  Yes  (Y) .  If  a  study  or  model  omits, 
provides  insufficient  detail,  or  misstates  a  model  or 
procedure  it  is  assigned  a  No  (N) .  Sometimes  decisions  are 
hard  to  make  because  of  minor  flaws  that  would  not  warrant  a 
Yes  or  a  No.  In  this  case  an  *  is  assigned.  Pure  model 
presentations  will  not  be  affected  by  the  Sample,  Theory 
Based,  or  Evaluation  Method  criteria.  Not  applicable  (N/A) 
will  be  assigned  to  these  criterion. 

This  research  did  state  its  purpose  but  importance  was 
never  covered.  An  example  and  calculations  are  supplied  but 
critical  steps  are  missing  and  explanations  are  too  general. 
Without  an  adequate  example  it  is  difficult  to  do  the 
calculations.  Limitations  were  described.  One  limitation 
was  the  assumption  of  a  normal  distribution  for  cost  and 
schedule  data  (20:7). 

The  weaknesses  of  the  study  are  in  major  areas  that 
represent  good  research.  The  study  expected  a  very  good 
understanding  of  Bayesian  probability  by  the  reader.  Terms 
and  calculations  were  not  defined  or  adequately  described. 
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El-Scibban  associated  the  CPR  data  elements  to  the  formula 
elements  (See  page  45  for  association)  but  there  was  no 
explanation  of  why  the  particular  CPR  element  corresponded 
to  a  particular  formula  element  (20:8).  No  literature 
review  was  conducted.  The  model  was  not  validated. 
Generally,  this  research  left  many  questions  and  unexplained 
critical  elements  of  the  proposed  model. 

Karsch  Model  1974  (2)  ♦ 

Karsch  presented  a  model  based  upon  a  nonlinear  equation. 
The  terms  were  defined  and  the  data  source,  one  contract, 
was  identified.  The  data  were  provided.  There  was  some 
theory  based  (Y*)  research  (nonlinear  relationship  of  data) 
but  as  an  aside  not  as  a  main  point  and  no  elaboration  of 
how  many  data  points  were  obseirved  to  reach  the  conclusion 
(28:13).  The  study  also  theorized  that  '‘completed  program 
samples  are  similar  in  their  characteristics  and  Y  =»  b  ,  X 
is  a  reasonable  behavioral  relationship  ....  that  b  ;  and 
bj  reflect  these  similarities  "  (28:4).  The  similar 
characteristics  that  b  i  and  b  j  reflect  were  not  stated. 

One  of  the  weaknesses  of  this  study  is  that  the 
conclusions  are  not  supported.  Karsch  refers  to  locking  at 
a  “variety  of  samples”,  and  b  j  was  found  to  be  “between 
1.18  and  .97“  but  there  is  no  supporting  details  (28:15). 
Karsch  also  concludes  the  model  is  an  improved  met.;od. 
However,  the  research  is  based  on  only  one  contract  so 
generalizing  to  other  contracts  is  not  justified.  The 
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purpose  is  implied  and  the  importance  of  the  study  is  never 
revealed.  According  to  Neter,  some  non  linear  parameters 
can  be  transformed  to  make  them  linear  (37:550-551).  There 
was  no  evidence  that  transformation  of  the  nonlinear 
parameters  was  attempted.  This  process  seems  to  be  a 
logical  step  for  this  research.  No  validation  of  the  model 
or  the  b  2  was  presented. 

Holeman  Model  fSK 

This  study  was  very  strong  in  Research  Clarity  and 
Documentation.  The  purpose  and  importance  were  straight 
forward.  Ha  viewed  the  use  of  EAC  formulas  as  a  more 
flexible  model  and  a  check  against  the  contractors'  EAC 
since  these  estimates  were  generally  optimistic  (27:5). 
Research  into  other  literature  on  EAC  formulas  was  performed 
and  presented  in  the  Appendix  (27: Appendix  A).  The  wording 
and  organization  aided  in  the  understanding  of  the 
procedures  and  basis  for  the  models  (3).  Examples  and 
associated  calculations  were  provided  which  also  helped  in 
understanding  the  models.  Conclusions  provided  limitations 
of  the  technique  and  of  forecasting  in  general. 

In  the  conclusions  Holeman  stated  that  baseline  changes 
greatly  affected  EACs  generated  by  historical  performance 
indices.  This  was  not  debated  or  tested  as  part  of  the 
research  design,  but  merely  referenced  as  an  advantage  of 
the  models  presented  since  the  models  were  not  solely  based 
on  historical  indices  (27:29-30).  This  is  why  a  **No"  was 
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given  to  the  "Theory  Base"  criterion.  The  experience  of  the 
analyst  and  the  amoiant  of  input  from  other  SPO  experts  were 
limitations  to  the  effectiveness  of  the  model  (27:31-32). 
These  models  were  not  validated  (use  actual  program  data)  in 
the  study  or  in  any  other  study. 

Busse  Model  fSK 

This  research  developed  a  model  that  was  based  on  the 
theory  that  a  relationship  exists  between  ACWP  and  BCWP  that 
can  be  developed  into  a  model  (8:22-23).  The  relationship 
is  for  forecasting  only  since,  in  reality,  more  factors  than 
BCWP  influence  ACWP  (8:23).  The  Research  Clarity  section 
was  given  a  Yes  for  the  criteria.  Cne  recurring  limitation 
in  the  model  development  research  is  that  the  model  not  be 
used  as  a  sole  predictor  of  EACs.  Busse  clearly  warns  the 
user  against  the  sole  use  of  his  model  (8:5).  Data  form 
only  one  contract  were  included  for  illustration  purposes. 
Accordingly,  "Sample",  "Representative",  and  "Data  Included" 
are  marked  "Yes"  and  "Size"  marked  "No".  H/A  was  assigned 
to  the  "Reproducible"  criterion  because  there  was  no 
comparison  of  formulas  or  validation  of  the  model. 

hevdinaor  (SAMSO)  study  (7). 

The  SAHSO  was  a  study  that  presented  a  modified  model  and 
was  also  a  comparative  study.  One  of  the  few  strengths  of 
this  study  is  that  no  conclusions  were  made  about  the  SAK^O 
model.  This  seems  like  a  strange  statement,  but  this  study 
had  limitations  in  its  design  and  to  formulate  any 
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conclusions  about  the  SAMSO  model  based  on  this  study  would 
have  degraded  the  integrity  of  the  study  (26:33).  The 
comparison  and  demonstration  was  done  using  one  data  progrcim 
which  is  not  adequate  to  draw  conclusions  about  model 
comparison.  The  data  were  provided  and  the  comparison  can 
be  reproduced.  The  study  refers  to  historical  data 
evaluation  but  does  not  reveal  the  number  of  contracts  used. 

Explanation  of  evaluation  methods  and  supporting 
documentation  were  strong  weaknesses.  The  statement  was 
made  that  "SAIISO  data  analyzed  to  date  shows  the  same 
similarity  with  falling  between  1.18  and  .97*'  but  no 
data  was  provided  to  support  the  statement  (26:6).  This 
statement  is  incorrect  because  it  is  b  ^  not  bj  that  falls  in 
the  range  (28:15).  The  misstatements  and  lack  of 
explanation  and  literature  support  of  equations  gives  the 
appearance  of  personal  opinion  and  hurried  research.  Lack 
of  explanation  of  ’’closest  to  final  cost”  for  picking  the 
formula  is  just  one  exampia  (26:10-11). 

The  description  of  the  procedure  and  basic  explatiation  of 
the  approach  is  hard  to  follow,  degression  analysis  was 
used  and  statistical  problems  like  autocorrelation  and 
heteroscedasticity  were  checked  for  and  resolved  as  much  as 
possible  (47:7-9).  A  sample  of  22  R&D  contracts  is 
considered  too  small  to  generalize  to  all  R60  cor'-racts. 
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The  study  is  reproducible  because  the  programs  are  listed 
and  a  little  research  will  produce  the  data  (47:56). 

No  conclusions  were  presented,  just  a  model.  No  limitations 
were  mentioned  in  the  study. 

hollar  Model  flO) . 

hollar's  model  was  a  project  that  presented  a  different 
technique  with  no  justification  or  explanation.  This 
project  presented  a  model  only.  The  "Purpose”,  "Define 
Terms",  "Importance",  and  "Calculations"  were  all  stated  but 
the  research  did  not  continue  past  this  point. 

The  "Conventional  Model"  was  confusing  because  it  appears 
as  if  the  formula  is  a  weighted  index  formula  but  later  in 
the  explanation  it  is  BAG  /  CPI  {32:lla).  There  was  no 
evidence  of  a  literature  review  to  see  if  this  procedure  had 
already  been  developed.  There  was  no  supporting  explanation 
of  how  he  developed  the  weighting  procedure  or  the  logic 
behind  the  procedure.  Tahing  the  absolute  value  ignores  the 
direction  and  effect  of  the  variances.  The  absolute  value 
treats  the  variances  as  all  positive  when  in  reality  there 
may  be  a  negative  variance  or  both  variances  could  be 
negative. 

The  mathematical  procedure  (taking  absolute  value)  does 
not  appear  to  be  a  viable  way  to  determine  a  weight  percent 
for  the  performance  factor.  This  formula  was  validated  in 
Blythes'  1982  comparison  study.  The  comparison  resulted  in 
recommendation  for  discontinued  use  of  the  model  (6:8). 
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Chacko  Ai 

The  method  is  internal  to  the  MESGRO  computer  program. 

The  approach  works  on  the  theory  that  instead  of  fitting  the 
data  to  a  model  the  opposite  is  done  (9:75).  The  study  was 
understandable,  limitations  were  stated,  conclusions 
supported  by  the  data  presented,  and  terms  were  defined. 
There  are  limitations  in  that  it  predicts  well  in  the  short¬ 
term  but  only  predicts  the  direction  of  the  change  in  the 
long  run  (9:95) . 

Although  the  study  is  strong  in  the  Research  Clarity 
area,  it  is  weak  in  the  Documentation  and  Methodology  areas. 
It  did  use  a  database,  but  no  indication  of  the  source  of 
the  database,  the  calculations,  or  the  data  type  were 
provided.  .Thus,  it  is  not  reproducible.  The  approach  uses 
a  computer  program  called  MESGRO  so  calculations  were  not 
supplied  (9:81).  Without  seeing  the  internal  calculations 
it  is  very  hard  to  give  any  confidence  to  the  methodology. 

No  formulas  are  available  for  this  approach. 
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since  the  "adjustment  factor"  was  designed  for  the  26 
program  mix,  the  forecasts  limitation  should  be  the  dollar 
range  of  the  programs.  This  limitation  is  not  addressed  and 
the  implication  in  the  study  is  that  the  factor  can  be  used 
universally.  A  set  of  different  programs  was  not  used  to 
validate  the  model. 

Appendix  A  of  Lollar's  report  was  lost  in  the  library 
system.  This  section  may  have  contained  detailed 
calculations  and  an  explanation  of  procedures.  Accordingly, 
a  "No"  rating  is  given  for  the  "Understanding", 
"Calculations",  and  "Reproducible"  criteria.  "Data  Source 
Stated"  and  "Data  Included"  criteria  are  each  given  a  "Y" 
because  they  are  included  in  his  referenced  1982  study  and 
were  used  to  establish  the  adjustment.  There  has  been  no 
validation  of  this  formula. 

■Tot aro  Model  (191.. 

The  Totaro  approach  was  one  of  the  few  reports  published 
in  a  periodical.  It  had  a  clearly  stated  purpose. 
Definitions  and  the  "Importance"  were  also  revealed. 
Calculations  and  examples  were  provided  so  manipulation  and 
understanding  the  working  of  the  formula  was  easy.  This 
study  used  six  contracts  for  a  comparison  with  the  best 
formula  of  Prices*  study.  Numerical  results  or  conclusions 
from  the  comparison  were  not  stated just  obvious 
generalities  (45:33). 
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The  source  of  the  data  was  stated  but  the  program  names 
were  not  included  so  Sample  and  Reproducible  criteria  were 
assigned  "Ns".  Description  of  the  procedure  was  good  but 
justification  of  his  linear  relationship  of  SPI  and  cri  to 
percent  complete  was  not  given  (Y*  in  Description) .  There 
was  no  evidence  of  any  review  of  literature  to  support  his 
assumptions  or  theories.  This  method  did  not  include  a 
sample  size  sufficient  for  validation. 

Model /Approach  Summary 

This  analysis  concentrated  on  studies  that  were  developed 
or  modified  models  and  new  approaches.  Some  criteria  were 
considered  not  applicable  to  the  study  because  the  study 
presented  a  model  only  with  not  comparison  to  other  models 
or  examples.  The  next  section  will  review  comparison 
studies  which  have  as  their  major  focus  the  comparison  of 
different  models. 

Analysis Comparison  Studies 

The  following  studies  compare  different  models  in  order 
to  select  the  most  accurate  predicting  model  of  final  cost. 
Some  studies  present  new  models  along  with  the  comparison. 
Some  of  the  methodologies  used  in  the  research  did  not  use 
regression  analysis  or  any  statistical  evaluation  as  a  part 
of  the  decision  criteria.  If  this  is  the  case,  "Statistics 
Tested"  criterion  will  be  assigned  a  Mot  Applicable  (N/A) 
rating.  The  same  evaluation  terminology  (Yes  and  No) 
applies  to  these  studies. 
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Karsch  (1976)  Study  r  4  K 

The  study  was  a  sequel  to  his  study  done  in  1974  with  the 
definitions  and  explanations  referenced  to  that  study.  The 
exception  is  Purpose  and  Importance  which  were  addressed  in 
tliis  study.  Calculations  for  a  missile  and  aircraft  program 
were  supplied. 

The  evaluation  criteria  for  closest  to  the  final  cost  is 
still  not  explained  (N  for  Description) .  Results  were 
mentioned  in  the  study  with  no  support  or  literature  review 
supporting  the  formula  or  assumptions.  Data  for  one  sample 
of  aircraft  and  missile  were  included  but  the  source  of  the 
rest  of  the  data  was  not  included.  There  was  no  indication 
of  aircraft  or  missile  number  but  sample  size  is  very  small 
for  conclusions  of  b  i  value  ranges.  Since  the  formula  was 
nonlinear,  no  testing  was  done  to  see  if  transposing  would 
produce  a  linear  relationship  (N  Test  Statistical 
Assumptions) . 

This  was  a  follow-on  study  of  Bayesian  probability  done 
by  El-Sabban.  Research  Clarity  and  Documentation  were 
strong  points.  Purpose,  Importance,  and  Definition  of  Terms 
met  the  criteria,  but  in  order  to  understand  and  follow  the 
study  a  thorough  knowledge  in  Bayesian  probability  was 
needed  (Y*  for  Understandable) .  The  same  limitations  as  to 
assumptions  as  £1-Sabban  were  recognized.  The  conclusions 
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were  supported  with  adequate  explanations.  The 
Docmentation  was  available  to  reproduce  the  study. 

The  major  problem  is  the  sample  size  (5) .  Statistical 
assumption  tasting  was  not  applicable  (N/'A)  because  the 
evaluation  criteria  were  based  on  what  estimate  was  closest 
to  the  final  cost  (25:44) . 

Land  and  Preston  Study  f9) . 

This  study  did  look  at  or  add  to  EAC  theory  as  it  applies 
to  linear  and  nonlinear  equations  and  to  the  theory  that 
Karschs'  growth  exponents  (h  ^  and  bj  )  reflect  the 
characteristics  of  the  program  (aircraft)  by  exhibiting  a 
narrow  range  (30:10).  This  program  did  use  completed 
programs  but  the  explanations  and  organization  made  the 
study  hard  to  follow  at  a  few  points. 

The  weaknesses  of  this  study  are  in  the  Documentation  and 
Sample  criteria.  The  sample  consists  of  30  contracts  with 
20  being  aircraft  programs.  A  sample  of  30  contracts  is  an 
adequate  sample  for  the  first  part  of  the  study  that 
evaluated  the  ESD  models,  but  when  the  ESD  and  Karsch  models 
are  evaluated  only  20  aircraft  programs  are  considered 
(30:50).  The  population  was  defined  as  **311  DOD  procurement 
programs  which  require  contractor  submittal  of  either  a 
Cost/schedule  Status  Report  (C/SSR)  or  Cost  Performance 
Report  (CPR)"  (30:19-20).  since  the  database  is  not 
included  and  the  aircraft  names  are  unknown,  the  data  are 
suspected  to  be  Air  Force  only  since  ASD  Contract  Library 
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was  the  source  of  the  data  (30:21).  By  using  C/SSR  data, 
which  does  not  require  a  C/SCSC  validated  system,  the 
reliability  of  the  data  now  comes  into  question  and  affects 
the  Representative  criterion. 

The  documentation  is  very  poor  with  this  study.  Land  and 
Preston  transformed  the  Karsch  model  by  the  use  of 
logarithms.  Karsch  calls  his  model  a  nonlinear  model.  If 
this  is  truly  a  non  linear  relationship  then  transforming 
the  parameters  will  not  give  you  a  linear  relationship 
(37:551).  There  is  no  scatterplot  provided  that 
demonstrates  this  transformation  is  linear  (30:29).  The 
absence  of  the  database  makes  the  study  not  reproducible. 

No  limitations  were  provided.  The  results  imply  \':hat  they 
can  be  used  universally,  but  in  fact  the  database  was 
limited  to  Air  Force  programs. 

Covach_  {NAVSEA)  Study  fll)  . 

This  study  satisfied  all  the  evaluation  criteria.  Except 
for  size  of  data  base,  it  is  given  a  solid  Ves  (V).  This 
study  dealt  with  NAVAIR  contacts  only  so  the  population  was 
small  to  begin  with  (16).  Six  programs  were  used  in 
evaluating  the  formulas.  As  compared  to  the  population,  the 
six  programs  are  a  significant  percent  of  the  population 
(Y*) .  This  was  noticed  as  a  limitation  of  the  study 
(Hayson:3).  The  other  programs  were  used  in  the  manpower 
evaluation  (12:55).  Regression  and  Past  Performance  Factors 
were  not  compared  to  each  other. 
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The  extension  study  (Haydon)  was  not  evaluated  in  Table  3 
because  it  is  presented  as  added  information.  The  second 
report  did  not  use  the  same  nomenclature  as  the  original 
study.  CPI  was  defined  in  the  original  study  as  ACWP/BCWP 
and  in  the  extended  report  it  is  defined  as  BCWP/ACWP 
(24:5) .  Equations  are  restated  using  this  relationship  so 
accuracy  is  not  affected.  A  different  method  was  presented 
for  calculating  the  EAC  using  a  range  approach.  The  "EAC 
Range”  is  not  defined  as  to  the  formula  used  to  calculate 
the  EACs  (24:3-12) . 

Bright  and  Howard  study  (12). 

Research  Clarity  was  a  strong  area  in  the  study  (all 
Yes) ,  The  purpose  was  to  compare  formulas  that  are  provided 
in  a  computer  program  so  managers  could  pick  formulas  to  use 
that  were  better  for  different  percent  completion  points  of 
a  contract  (7:1-3).  Terms  and  limitations  were  explained 
and  conclusions  were  supported  by  the  data  (7:13).  The  new 
model  was  based  on  the  theory  that  ”CPI  must  be  deflated 
when  behind  schedule  and  inflated  when  ahead  ”  (7:17).  CPI 
X  SPI  represents  this  relationship  (7;17). 

The  weak  area  of  the  study  was  in  Documentation.  The 
data  source  was  indicated  but  the  programs.  The  data  were 
not  included  and  the  calculations  to  support  the  CPI  «  x 
SPicu,  combination  was  not  provided  (7:8).  There  was  a 
review  of  literature  as  indicated  in  the  bibliography,  but 
very  little  citation  of  the  sources  in  the  text  (Y*  rating) . 
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Looking  at  Army  R&D  contracts  as  the  population  the  sample 
size  of  11  could  be  significant  (Y*) .  The  study  is  limited 
to  Army  contracts. 

Blvthe  ri982^  Study  fl4K 

This  is  one  of  the  studies  cited  by  Wallender  to  justify 
the  AFSC  weights.  This  study  used  26  progreuns  which 
included  missiles,  engines,  guns,  armament,  and  aircraft  (6: 
Appendix  A) .  It  did  not  evaluate  by  contract  or  production 
item  type,  so  sample  size  is  adequate  considering  the 
population  of  ASD  contracts  with  CPR  or  C/SSR  requirements. 
The  criterion  of  Representative  was  based  primarily  on  the 
reliability  of  data,  since  the  c/SSR  does  not  require  a 
validated  C/SCSC  system,  the  C/SSR  data  are  suspect. 

Because  this  study  included  C/SSRs  it  is  rated  Y*.  The 
Purpose  of  the  study  was  to  validate  a  model  that  was  being 
used  by  analysts  and  ASD.  This  study  was  reproduced  by 
Cryer  and  Balthazar. 

This  study  was  strictly  a  comparison  of  £AC  formulas.  No 
new  £AC  formulas  were  presented.  No  new  theory  was 
proposed.  It  looked  at  different  percent  completion  points 
and  evaluated  models  using  percent  accuracy  and  standard 
deviation.  The  conclusions  imply  a  generalization  to  any 
contract,  but  this  is  risky  since  the  sample  had  few 
contracts  of  different  types.  There  were  no  limitations 
stated  in  the  study.  Regression  assumptions  were  not  tested 
or  stated. 
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A  separate  part  of  the  study  looked  at  the  different 
weighing  %  that  could  be  used  with  the  Parker  Model.  The 
weighing  evaluation  was  based  on  the  lowest  Coefficient  of 
Variation  (CV)  (6:5) .  There  were  no  calculations  in  the 
appendix  to  support  the  conclusions  presented  in  this  area 
(Y*  for  Conclusions) . 

Price  study  (16). 

Prices'  Master  Thesis  is  a  purely  comparative  study. 

Price  states  his  purpose  of  evaluating  the  six  EAC 
techniques  is  to  find  a  the  most  accurately  predicting 
formula  within  the  CPRA  analysis  program  (40:3,13).  The 
conclusions  do  relate  to  the  data  results  and  the 
limitations  of  the  study  were  state. 

Weaknesses  of  this  study  are  in  the  Documentation  and 
Methodology  sections.  There  was  no  indication  of  any  type 
of  literature  review  except  for  quotes  from  one  past  study. 
The  Price  study  can  not  be  reproduced  because  the  program 
data  or  names  used  in  the  database  are  not  included.  The 
assumptions  for  regression  analysis  were  tested  but  no 
example  of  data  calculations  to  support  the  linearity 
assumptions,  normality,  and  homoscedasticity  are  provided 
(40:20-21). 

One  of  the  weaknesses  in  the  Methodology  section  was  the 
treatment  of  contracts  with  different  percent  completions  as 
if  they  are  finished  programs  at  that  percent.  No  completed 
programs  were  used.  The  last  CPR  BCWP  was  set  as  the  BAC 
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and  the  ACWP  was  set  as  the  final  cost  at  any  %  complete 
point  (40:17).  This  assumes  a  linear  relationship 
throughout  the  contract  at  all  stages  which  would  give  a 
worst  case  estimate  in  the  beginning  of  a  contract. 

Wallender  Paper  fl8K 

The  paper  by  Wallender  (20)  was  a  justification  of  the 
•'HQ  AFSC  EAC"  formula.  The  three  studies  that  justified  the 
formula  are  helpful  as  sources  of  EAC  formulas  and  results. 
Of  the  three,  only  one  study  could  be  found,  the  ASD  Reserve 
Study.  Wallenders'  paper  is  omitted  from  Table  3  because  it 
is  not  a  study.  In  reading  the  summary  of  the  HQ  AFSC 
Study,  th6  methodology  was  to  compare  an  estimate  to  an 
estimate  (46:2).  Without  more  details  this  method  seems  to 
evaluate  how  close  the  formula  is  to  another  estimate,  not 
the  final  cost. 

The  AD_Studv  compares  final  contract  cost  to  the  EAC  from 
the  AFSC  formula,  but  then  compares  the  OSO  EAC  and  AFSC 
EAC.  The  OSD  EAC  was  found  to  be  10-124  higher  than  the 
AFSC  EAC  but  there  is  no  indication  as  to  how  the  OSD  EAC 
formula  compared  to  final  costs. 

Balthazar  and  Crver . Study  f 17). 

The  study  is  a  reproduction  of  the  previous  study  and 
contains  the  calculation  which  can  be  used  to  verify  results 
{ 13: Appendix) .  The  study  is  a  replication  of  Blythes’  1982 
study  so  ratings  and  reasons  for  sample  and  Evaluation 
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Method  are  the  same.  The  study  looks  at  practice  rather 
than  theory. 

Weaknesses  include  Conclusions  and  Limitations  criteria. 
The  findings  and  recommendations  are  hard  to  interpret  since 
no  evaluation  criteria  were  stated.  The  weightings  of  .1 
and  .9  as  well  as  the  .2  and  .8  percentages  were  very  close 
according  to  the  coefficient  of  variation  (CV) .  Conclusions 
state  the  weights  of  .4  and  .6  had  the  lowest  standard 
deviation,  a  point  that  had  no  significance  since  the 
evaluation  criterion  is  lowest  CV.  Regression  analysis  was 
used,  but  no  assumption  testing  was  performed  or  presented. 
The  Lollar  model  was  also  stated  incorrectly.  In  the  Cryer 
and  Balthaser  study,  the  model  was  written  as  the  Schedule 
and  Cost  variance  factor  multiplied  by  the  absolute  value  of 
the  Schedule  or  Cost  performance  index  which  is  incorrect. 
The  SVf  and  CVf  were  also  calculated  incorrectly  because  the 
absolute  value  of  SVP  and  CVp  are  added  together  for  a  total 
percent  then  the  SVP  or  CVP  are  divided  by  the  total  (13:3). 
Prom  the  calculations  it  appears  this  formula  was 
miscalculated  which  could  change  the  comparison  results  but 
not  the  weighting  combination  comparison. 

Reidel.  and  Chance  Study  (2QL. 

This  is  one  of  the  most  recent  comparative  SAC  study. 
Formulas  were  evaluated  according  to  different  percent 
completion  points,  contract  types,  and  end  Item  types.  The 
study  is  reproducible,  understandable,  and  includes 
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calculations  and  explanations  of  all  data  sources  (41:A-1  - 
A-15)  . 

The  definition  of  EAC  7  is  not  comparable  to  the  actual 
formulas  referenced  (41:5-5,12-13).  The  Totaro  model  does 
use  percent  complete  for  determining  a  weighting  factor  but 
not  as  stated  in  this  study.  The  SPI  weighting  is 
determined  by  the  weight  decided  by  the  analyst  then  reduced 
by  a  factor  based  on  percent  complete.  The  CPI  weighting  is 
increased  in  the  same  manner. 

Totaro  factors: 

Wght  SPI  =  .25-C .25(%complete)SPI]  (66) 

Wght  CPI  =  .75-^C*25(%complete)CPI)  (67) 

Example:  30%  complete 

.25-(.25(30%)SPI)  ■»  .2S-7.S  ^  .175 

.75+ (.25 (30%) CPI)  “  .75+7.5  »  .825 

(45:31) 

Reidel  Study  EAC  7: 

X  “  %  complete  »  30% 

X  (CPI)  +  (1-x)  SPI  «  .3CPI  +  .7SPI  (67) 

Because  these  methods  are  not  the  same,  the  **Terms  Defined'* 
criterion  is  awarded  a  Ho  (H) . 

The  study  did  not  use  citations.  Ho  limitations  were 
stated  but  the  OSMC  course  svaterial  formulas  came  from  the 
NAVSEA  study.  The  HAVSEA  study  has  a  limitation  in  the  fact 
that  the  database  used  to  evaluate  the  formulas  are  Havy 
programs  only  (12:5).  The  sample  size  is  small  for  each 
contract  type  and  end  item  but  could  be  appropriate 
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considering  the  population  is  ASD  contracts  (Y*) .  The 
methodology  uses  "Average,  Absolute  Value  Deviation"  method 
as  the  primary  evaluation  criterion  and  "Average  Rank  Order" 
as  a  check  (12:18“-19).  No  statistical  assumptions  needed  to 
be  tested  (N/A) . 

Summary 

This  chapter  reviewed  the  actual  studies  by  categories  of 
model  and  comparison  studies.  The  studies  were  evaluated  on 
clarity,  documentation,  and  methodology.  Results  of  the 
analysis  will  help  evaluate  the  study  results  based  upon 
the  strengths  and  weaknesses  of  the  studies.  The  next 
chapter  will  review  the  objectives  of  this  researcii  and 
provide  recommendations  for  future  research. 
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IV.  conclusions  an<A  Resoinroenda_ti_ons. 

The  A-12  Program,  tighter  DGD  budgets,  increased 
oversight  by  congress  and  DOD,  and  the  reduction  in  manpower 
are  bringing  the  role  and  use  of  EAC  formulas  to  a  heighten 
level.  Program  and  Financial  managers,  as  well  as,  CPR  and 
C/SSR  analysts  will  not  be  able  to  ignore  the  estimates 
generated  by  the  EAC  foxrmulas.  The  increased  attention  EAC 
formulas  will  achieve  in  the  future  will  increase  the 
research  into  this  area.  Because  of  this,  knowing  what  has 
been  accomplished  is  essential. 

In  Chapter  I  the  four  objectives  of  this  research  were 
proposed  in  order  to  accomplish  the  above.  The  conclusions 
will  be  structured  around  these  objectives. 

Fir3.t__Qbiective  t Identify  and  Col  1  ect ion 

In  order  to  establish  any  research  base  the  researcher 
must  know  where  to  look  for  the  literature.  When  this 
research  was  started,  it  was  a  belief  that  very  little 
research  had  been  done  in  the  EAC  area.  This  belief  was 
soon  disproved  because  24  different  sources  were  found. 
Finding  these  studies  was  not  easy. 

Host  of  the  sources  were  obtained  through  the  Defense 
Technical  Infonoation  Center  (OTIC)  system  or  at  the 
Aeronautical  System  Division  (ASD)  Cost  Library  at  Wright- 
Patterson  AFB.  Talking  to  people  in  the  C/S  area  and 
searching  the  bibliographies  of  obtained  research  was  the 
main  technique  used  for  r'  .i:  search.  The  belief  of  this 
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researcher  is  that  more  EAC  studies  are  buried  in  libraries 
at  Navy  and  Army  installations.  Any  research  should  involve 
a  review  of  the  libraries  along  with  the  OTIC  and 
bibliography  searches. 

It  is  obvious  that  the  dissemination  of  this  knowledge  to 
the  field  is  very  limited  and  unpublished.  Only  two  of  the 
studies  were  found  in  professional  journals  or  publications. 
No  wonder  Program  and  Financial  managers,  as  well  as  the  C/S 
analysts,  are  not  aware  of  the  different  types  of  EAC 
formulas  and  give  little  confidence  to  the  results  from  EAC 
formulas. 

Second  Objective:  Categorization  and  Summarization 
This  objective  starts  the  foundation  building  of  the  EAC 
research.  The  research  studies  start  in  1973  and  focusedthe 
on  model  development  until  1977.  A  few  studies  during  this 
period  did  comparisons  but  a  new  approach  or  model  was 
usually  presented.  These  authors  were  the  pioneers  in  the 
EAC  area.  The  focus  during  this  time  period  was  on  model 
building  and  new  approaches. 

From  19P0  to  1986,  Blythe,  Lollar,  Cryer  and  Balthazar, 
and  Wallender  focused  on  the  best  combination  of  percentages 
for  weighting  indices.  Lollar 's  formula  was  used  for  a 
period  of  two  years  or  longer  at  ASD  before  Blythe's  study 
showed  that  it  should  be  replaced.  The  perception  is  that 
EAC  work  before  Lollar  was  not  critically  analyzed  but  just 
accepted.  During  this  time  period,  the  trend  changed  from 
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model  development  to  evaluating  the  forecasting  ability  of 
the  EAC  formulas. 

In  the  last  five  to  six  years  the  emphasis  has  been  on 
the  comparison  of  EAC  formulas.  The  studies  focused  on 
Index  type  formulas  and  usually  involved  the  same  group  of 
formulas.  Regression  models  have  been  reviewed  but  little 
work  has  been  done  on  this  category  in  the  last  10  years. 
There  have  been  many  other  approaches  such  as  Time  Series, 
Bayesian  probability,  S-Curve,  logic  formulas,  and 
exponential  relationships  that  have  been  developed  but 
little  work  has  been  done  to  expand  upon  or  validate  these 
models  or  approaches.  This  may  leave  the  analyst  and  other 
interested  personnel  with  the  belief  that  there  are  only  a 
handful  (4-5)  of  formulas  that  can  be  used  to  generate  an 
EAC.  In  reality,  there  are  an  infinite  number  of  EAC 
formulas.  Appendix  C  shows  43  EAC  formulas  reviewed  in  this 
thesis. 

Third  Objective;  .  Analysis  o,f_Studies 

In  order  for  managers  to  have  confidence  in  the  EAC 
formulas  the  research  itself  must  be  done  in  a  convincing 
manner  to  instill  confidence  in  the  results.  Of  the  19 
studies  analyzed  in  Table  3,  the  following  weaknesses  were 
identified: 

1.  Limitations  not  stated 

2 .  Documentation  inadequate 

A.  Calculations  missing 
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B.  Data  not  included 

C.  Size  of  sample 

D.  Description  of  Data 

E.  Reproducability 

3.  Methodology  defective 

A.  Statistical  parameters  used 

B.  Proof 

4.  Literature  review  deficient/ absent 

5.  Lack  of  theory  based  studies 

The  “Limitations'*  of  a  study  are  critical  to  how  the 
results  are  applied.  One  of  the  major  limitations  was  that 
results  applied  to  a  certain  database  (Army,  Navy,  Air 
Force)  were  being  generalized  to  a  broader  population.  A 
cost  analyst  generalizing  results  to  an  Air  Force  program 
that  was  generated  from  a  Navy  database  would  have  a  tough 
time  justifying  the  EAC  formula  or  the  estimate. 

“Documentation"  was  the  weakest  category.  The  major 
problem  was  reproducability.  Having  a  study  that  can  be 
reproduced  (results  and  procedures)  by  other  researchers 
adds  validity  to  the  conclusions  and  future  work  of  that 
researcher.  Missing  calculations  to  support  results, 
databases  that  were  considered  to  small  to  be 
representative,  and  description  of  why  the  data  used  were 
accurate  and  reliable  were  areas  that  also  contributed  to 
poor  documentation.  Not  including  or  identifying  the  data 
sources  or  contracts  used  makes  it  impossible  to  reproduce  a 
study . 
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Another  area  that  goes  along  with  documentation  was  the 
lack  of  methodology  description.  Some  studies  did  not 
explain  the  decision  criteria  for  all  the  results  presented. 
In  the  earlier  studies  (1973-1981),  statistical  procedures 
(regression  analysis,  standard  deviation,  coefficient  of 
variation)  were  used  without  any  explanation  of  "why".  In 
some  cases  the  "why"  would  be  explained  but  proof 
( scatterplots ,  graphs,  calculations)  of  the  conclusions  of 
the  tested  assximptions  was  not  supplied. 

One  area  of  surprise  was  the  lack  of  a  literature  review. 
Of  the  19  studies,  nine  had  no  bibliography,  or  review  of 
prior  studies,  or  citations  to  support  the  data  used.  This 
supports  the  idea  that  research  in  the  EAC  area  is 
undisciplied  and  lacks  direction.  Theory  based  EAC  research 
was  found  to  be  rare. 

It  is  not  concluded  that  the  EAC  research  lacks  merit. 

But  for  the  research  to  be  accepted  by  practitioners  and 
managers,  it  must  be  able  to  withstand  a  critical  review. 
Confidence  in  results  and  solid  research  that  can  be  used 
for  justification  by  an  analyst  or  manager  is  a  must  for  EAC 
formulas  to  be  actively  considered  and  used. 

Fourth  Objective;  Study  Results 

What  can  be  gleaned  form  all  of  this  research?  One 
"sacred"  EAC  formula  was  that  .2  SPI  and  .8  CPI  was  the  best 
combination  of  weighted  indices.  When  an  analysis  of  the 
Blythe  and  Cryer  was  done,  this  long  standing  standard 
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becomes  a  little  shaky.  The  weaknesses  and  strengths  in  the 
studies  lead  to  the  conclusion  that  .1  SPI  and  .9  CPI,  .2 
SPI  and  .8  CPI,  and  .3  SPI  and  .7  CPI  are  combinations  that 
yield  equally  accurate  EACs. 

Managers  (progreua  and  financial)  usually  review  the 
contractor's  EAC  with  skepticism.  However,  the  early 
literature  showed  this  EAC  to  be  the  most  consistent  and 
accurate.  Blythe  showed  this  in  his  study  and  developed  an 
EAC  formula  that  utilizes  the  contractor's  EAC.  One  result 
that  all  the  researchers  agreed  with  was  that  more  than  one 
EAC  formula  should  be  used  to  develop  an  EAC  and  the 
possibility  of  using  the  contractor's  EAC  could  be  one  of 
the  methods. 

When  the  different  types  (Past  Performance  Factors  (PPF) 
and  Regression)  of  formulas  are  reviewed  the  area  of  least 
emphasis  has  been  in  the  regression  area.  Bright  and  Howard 
did  some  work  with  two  regression  models  in  comparing  them 
with  PPF  type  formulas  and  found  that  up  to  30%  complete  the 
regression  formulas  did  well  at  predicting  final  cost.  The 
NAVSEA  study  reviewed  a  number  of  regression  models  but 
evaluated  them  against  each  other  not  against  PPF  type 
formulas.  This  is  still  a  wide  open  area  for  research. 

An  analyst  usually  uses  a  PPF  type  of  formula.  The  work 
in  this  area  has  compared  formulas  by  certain  completion 
points,  by  type  of  contract,  and  by  product  type.  Each 
study  showed  different  formulas  depending  upon  the  group  of 
formulas  used.  The  largest  group  of  formulas  was  done  in 
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the  NAVSEA  study.  An  analyst  must  evaluate  the  program  and 
the  study's  limitations  and  pick  the  EAC  formulas  that  are 
appropriate.  The  results  and  formula  rankings  are  presented 
in  Chapter  2. 

One  of  the  formulas  that  is  referenced  when  C/S  people 
talk  about  EAC  formulas  is  the  Howard  Model  (SPI  x  CPI) . 

Most  of  the  presentations  seen  by  this  author  show  this 
formula  erroneously.  The  model  was  originally  established 
to  be  the  Cum  SPI  multiplied  by  a  six  month  moving  average 
CPI. 

The  review  of  the  studies  showed  that  some  rules  of  thumb 
are  not  supported  by  the  literature.  The  weighting  of 
indices  is  still  a  wide  open  area.  Regression  analysis  is 
an  area  of  limited  research.  The  Karsch  model  shows  promise 
but  the  B  2  value  must  be  narrowed  down  for  product  type  and 
contract  type.  The  past  performance  index  models  have  some 
good  results,  but  the  limitations  of  the  studies  need  to  be 
described  by  the  researcher  and  considered  by  the  analyst 
when  doing  research  or  selecting  an  EAC  formula. 

The  data  relationship  is  another  area  that  needs  to  be 
cleared  up.  Most  studies  have  assumed  linearity.  Weida  and 
Karsch  showed  Cost/Schedule  data  to  be  nonlinear.  This  is  a 
basic  characteristic  that  needs  to  be  resolved. 
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Another  area  of  weakness  is  theory  based  EAC  research. 
There  seems  to  be  no  consideration  of  "why”  models  predict 
the  way  they  do. 

Recommendations  for  Future  Studies 

Instead  of  evaluating  formulas  by  contract  type  or 
product,  maybe  the  formulas  should  be  compared  by 
contractors.  Contractors  in  the  defense  industry  have 
different  management  styles  which  nay  be  reflected  in  the 
C/S  data.  An  evaluation  by  contractor  may  show  different 
formulas  are  best  for  different  contractors. 

Another  area  is  the  databases  used.  If  results  are  to  be 
applied  on  a  universal  level,  the  formulas  should  be 
evaluated  using  a  OOD  database  such  as  the  DAES  report. 

This  database  was  used  by  Scot  Heise  in  a  Masters  Thesis  in 
1991  at  AFIT  and  is  available  through  his  advisor.  Major 
David  Christensen  AFIT/LSY.  Another  area  that  goes  along 
with  the  database  is  the  testing  of  larger  groups  of 
formulas,  such  as  the  NAVSEA  study.  The  studies  should 
include  unvalidated  models  and  regression  models. 

Some  of  the  models  such  as  the  Bayesian  approach,  Busse 
Model,  Holeman  Models,  and  Weida  Model  can  be  very 
complicated.  The  computer  knowledge  and  ability  that  is 
currently  available  in  programming,  could  be  used  to  develop 
the  above  models  so  that  an  analyst  only  inputs  CPR  data  and 
gets  an  EAC  back. 
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Theory  should  be  developed  and  tested.  Payne  looked  at 
CPI  stcibility  in  a  1990  AFIT  thesis.  What  about  SPI 
stability  and  influence? 
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Appendix  A;  Explanation  of  Cost /Schedule  Control  Systems 
Criteria  (C/SCSC) . 

Coat/Schedule  Control  Systems  Criteria  fC/SCSC^ 

C/SCSC  is  not  a  meuiagement  system  imposed  upon  the 
contractor,  but  a  set  of  standards  that  the  government  uses 
to  evaluate  the  management  system  of  the  contractor  (16:V) . 
The  criteria  provide  a  set  of  minimum  standards  that  a 
contractor's  management  control  systems  must  comply  with. 
Organization,  Planning  and  Budgeting,  Accounting,  Analysis, 
Revision  and  Access  to  data  are  the  areas  the  Criteria 
address  (16:2-1,  2-3).  A  checklist  of  questions  is  used  to 
evaluate  if  the  contractors  management  system  complies  to 
the  criteria  (16;E-1,  E-16) .  After  a  successful 
demonstration,  the  management  system  should  provide  data 
which: 

1.  Indicate  work  progress; 

2.  Properly  relate  cost,  schedule  and  technical 
accompl ishment ; 

3.  Are  valid,  timely,  and  auditable; 

4.  Supply  managers  with  information  at  a 
practical  level  of  summarization. 

(13:5) 

In  order  to  maintain  confidence  that  the  management 
system  is  operating  as  demonstrated,  periodic  surveillance 
of  the  system  is  required.  Representatives  from  the  Defense 
Logistics  Agency  (DLA) ,  the  Contract  Administration  Office 
and  the  Program  Office  visit  the  contractor  to  make  sure  the 
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system  is  still  operating  as  it  was  validated  and  to  look  at 
problem  areas  that  appear  in  the  CPR  or  C/SSR  reports 
(17:1). 

Not  all  major  contracts  are  obligated  to  have  C/SCSC. 
Prime  contractors  that  receive  contracts  over  $60M  for 
Research  and  Development,  or  over  $250M  for  Production  are 
required  to  conform  to  C/SCSC.  Subcontractors  are  also 
required  to  have  their  management  system  compliant  to  C/scsc 
if  the  subcontract  is  determined  to  be  critical  by  the 
government  and  the  contractor.  (The  exception  is  a  Firm 
Fixed  Priced  (FFP)  contract)  (19:11-8-2  11-8-4).  If  the 

contractor's  management  control  system  is  C/SCSC  compliant, 
then  the  performance  data  generated  by  that  system  is 
assumed  to  be  reliable.  The  data  are  summarized  in  either 
the  CPR  or  C/SSR. 

The  data  are  accumulated  at  the  end  of  each  month  and 
transmitted  to  the  government  on  a  Cost  Performance  Report 
(CPR)  or  a  Cost/Schedule  Status  Report  (C/SSR) .  C/SSR 
reports  are  usually  for  contracts  that  do  not  require  a  CPR, 
are  over  $5  million,  and  are  at  least  one  year  in  length. 
C/SSR  reporting  does  not  require  a  system  that  is  C/SCSC 
compliant  (15:1). 

The  CPR  has  five  formats  and  the  C/SSR  has  two  formats. 

The  formats  are  described  below: 

CEB 

Format  1:  The  contract  is  structured  by  different 
product  items  or  segments  of  work.  Each  product  item  or 
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segment  requires  a  group  of  tasks  be  completed.  This  is 
called  a  Work  Breakdown  structure  (WBS)  and  is  similar  to  a 
pyramid  with  the  top  being  the  product  (airplane,  gun, 
missile) .  Each  iteration  is  termed  a  level  which  is 
numbered  from  the  top  to  the  bottom.  Format  1  reports 
cxirrent  and  cumulative  costs  by  the  WBS  structure  and 
usually  limits  the  structure  to  level  3. 

The  heading  shows  contract  information  such  as  contract 
price,  estimated  price,  authorized  but  yet  unpriced  work 
(estimated)  and  other  contract  related  elements.  There  are 
columns  that  show  variances  for  the  cost  and  schedule 
portions  as  well  as  a  column  for  the  contractors  Estimate- 
at'^Completion.  Management  Reserve,  Undistributed  Budgets, 
reprogramming  actions,  and  total  contract  variances  are  also 
shown  on  this  format. 

Format  2:  This  format  is  similar  to  Format  I  except  that 
the  cost  breakdown  is  by  functional  categories  such  as 
engineering,  quality  assurance  and  material.  Cost  and 
Schedule  Variances,  Estimate**at-Completion,  and  Management 
Reserve  are  also  shown  on  this  format. 

Format  3*  This  is  the  baseline  format.  It  shows  the  time 
phased  dollar  amounts  for  the  contract  life  at  the  total 
contract  level.  Changes  to  the  baseline  are  listed  and 
explained  with  the  bottom  line  showing  the  new  baseline 
dollars  and  new  time  phasing. 

Format  4:  The  manpower  for  the  program  is  time  phased  by 
functional  area  for  the  next  six  months  and  for  the  rest  of 
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the  progrcua  by  an  increment  determined  during  contract 
negotiations.  The  actual  manpower  for  the  reported  month  is 
presented  as  well  as  the  cumulative  amount. 

Format  5:  Significant  cost  and  schedule  variances  are 
explained  on  this  format.  There  are  usually  certain 
thresholds  set  for  reporting  on  the  format.  Corrective 
action,  areas  affected  by  the  problems,  correction  plans, 
and  dates  are  provided  for  functional  and  WBS  areas  that 
break  the  contract  threshold. 

C/SSR 

Format  1:  The  format  shows  the  cumulative  cost  by  WBS 
element  and  at  total  contract  level.  Contract  data  such  as 
cost,  negotiated  changes  and  estimated  cost  of  unpriced  work 
are  shown  in  the  heading.  Except  for  reprogramming  actions, 
all  other  elements  are  the  sane  as  on  the  CPR  Format  1. 

Format  2:  Explanation  of  the  problems  and  analysis  of 
what  can  be  done  about  them  is  the  major  function  of  this 
format.  This  is  similar  to  Format  5  in  the  CPR  and  can  be 
triggered  by  a  threshold  amount  {\  variance,  dollar  amount 
or  both) . 

The  data  elements  included  in  the  CPR  and  C/SSR  formats 
are  names  unique  to  the  C/SCSC  arena.  EAC  formulas  utilite 
these  elements  and  names. 

Budgeted  Cost  of  Work  Scheduled  (BCWS)  or  "planned  value" 
is  the  beginning  budget  allocated  to  tasks  in  the  WBS.  This 
allocation  is  done  at  the  beginning  of  the  contact  or 
effort.  This  is  the  starting  point  or  baseline  from  which 
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performance  is  evaluated.  The  budgets  are  allocated  over 
the  life  of  the  contract  and  can  be  detailed  planned  or  kept 
as  a  lump  sum  (planning  package)  until  the  start  date  of  the 
work  gets  closer. 

Budgeted  Cost  of  Work  Performed  (BCWP)  is  often  referred 
to  as  "Earned  Value".  Earned  Value  is  the  evaluated  amount 
of  work  completed  compared  to  what  was  scheduled  (BCWS) . 
Different  methods  for  evaluating  the  amount  of  work 
accomplished  can  be  used  but  they  must  be  similar  to  the 
methods  used  to  plan  BC\vS. 

Actual  Cost  of  Work  Performed  (ACWP)  refers  to  the  work 
performed  (BCWP)  not  the  work  scheduled  (BCWS) ,  The  actual 
direct  costs  (labor, parts, material)  and  indirect  costs 
(overhead)  are  represented  by  this  term. 

Management  Reserve  (MR)  is  usually  a  percentage  withheld 
from  the  contract  total.  This  amount  is  for  unexpected 
events.  The  program  manager  controls  this  budget  and  gives 
approval  for  its  use. 

Budget  at  Completion  (BAC)  is  the  budget  allocated  to  the 
contract  excluding  Management  Reserve  and  profit.  As 
contractual  changes  take  place  this  amount  will  change. 

There  are  two  frequently  used  ratios  in  the  evaluation  of 
the  contractors'  performance  and  in  the  EAC  formulas.  These 
ratios  are  called  Cost  Performance  Index  (CPI)  and  Schedule 
Performance  Index  (SPI) . 

The  CPI  is  the  ratio  of  BCWP/ACWP.  This  is  called  an 
efficiency  CPI  and  can  be  interpreted  as  the  dollar  amount 
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of  work  done  for  eacn  dollar  spent.  The  calculation  can  be 
done  for  current,  cumulative,  and  average  data.  Another 
CPI,  called  a  performance  CPI,  is  the  inverse  of  the 
efficiency  CPI.  The  efficiency  CPI  is  the  ratio  used  in  EAC 
formulas  and  will  be  referred  to  as  the  CPI  in  this  study. 

A  CPI  with  a  value  under  l.O  indicates  a  cost  overipin  on  the 
contract,  A  CPI  above  1.0  indicates  a  cost  underrun. 

The  SPI  is  calculated  by  BCWP/BCWS.  The  status  of  the 
schedule  is  represented  by  this  index.  An  index  less  than 
1.0  indicates  the  contract  is  behind  schedule  and  above  1.0 
indicates  an  ahead  of  schedule  situation.  This  index  can  be 
calculated  using  current,  cumulative,  and  average  data. 
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Appendix  B:  Formulas 


Indices 

Efficiency  CPI:  BCWP/ACWP 

Performance  CPI  (CPI p  ) :  ACWP/BCWP 

SPI:  BCWP/BCWS 

The  indices  can  be  used  with  current  or  ciamulative  data. 

CosjLVaiLUnGgs 
Schadule  Variance: 

Schedule  Variance  Index  (SVI) : 

%  Schedule  Variance: 

Cost  Variance: 

Cost  Variance  Index  (CVI) ; 

%  Coat  Variance: 

Contract  Budget  Base  (CBS) : 

CBS  *  Negotiated  Contract  Estimated  Unpriced  Work 
CBB  =•  BAC  +  MR 
BAC  excludes  MR 
BAC  »»  Performance  Baseline 

BAC  *  BCWPj^  =*  BCWR  (Budgeted  Cost  of  Work  Remaining) 

%  Complete  »  SCWP*^  /BAC  x  100% 


SV  =  BCWP  -  BCWS 
SV/BCWS 

SV/PCW3  X  100% 

CV  »  BCWP  -  ACWP 
CV/BCWP 

CV/BCWP  X  100% 
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Moving  Averages 


CPI3  =  (CPIn  +  CPI  0-1  +  CPI  a-2  )  /  3  (Average  of  Ratios) 
m  =>  monthly  data 

C^3  =(BCWPo  +  BCWP0.1  +  BCWPo.2  )/(ACWP„  +  ACWPa-i  +  ACWP^.j  ) 

(Ratio  of  Sums  for  Efficiency  CPI) 
C^P3  =  (ACWPo  +  ACWP„.i  +  ACWPo.2  )  /  (BCWPo  +  BCWPo.^  +  BCWP0-2) 

(Ratio  of  Sums  for  Performance  CPI) 
CPI3  *  =  (BCWP„  -  BCWP„.3  )  /  (ACWP„  -  ACWP^.a) 
n  =  Cumulative  Month  Data 


y  » 

a  -i'  bX 

(Linear  Curve) 

Y  « 

ax 

(Power  Curve) 

V  » 

ae 

(Exponential  Curve) 

LnY 

»  a  +  b  Lh  X 

(Log  Curve) 

y  a 

a  ■•'bX  +  cX^ 

(Quadratic  Curve) 
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All  data  elements  are  cimulative  unless  stated  in  the 


formula.  Cur  =  Current  month  data 

Cost  Performance  Report  Analysis  fCPRAl  Program 
EACl  =  ACWP  +  (a\C-PCWP)/CPlE 
EAC2  =  ACWP  +  (BAC-BCFi»)/CPlE 

EAC3  »  ACwP  V  (BAC’-BCWi')/CPl3  *  (See  Appendix  B) 

EAC4  =  ACWP  t  ETC 

ETC  a  [100- (Cost  Var.  %)+. 7 5 (Schedule  Var.  %) ]  X 

BCWR/100 

EAC5  EACC  +  ETCS 

EACC  a  (.12  X  EACl)  +  (.64  X  EAC2)  ■i'  (.24  x  EAC3) 
ETCS  a  (months  behind  Schedule)  x  ACWP  Rate  x  .75 
ACWP  Rate  -  ACWP/Total  contract  completed  months 

(40 ‘.10-12) 


EAC6  »  ACWP  +  (BAC-BCWP)/  (.2SPI  +  .8CPI) 

(46:1) 


<3gp.-f:pin>ula 

EAC7  a  ACWP  +  (BAC-BCWP)/  (CPI) (SPI) 

(46:3) 
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^sis  Program 


Electonic  System  Division  (ESD^  Financial  Analv 
General  formula  EAC  =  ACWP  +  ETC 
EAC8  ETC  =  (1  -  CVIcm  )  BCWR 

(30:23) 

EAC9  ETC  =  [1  -  (CVI„  +  CVI„.i  +  CVI^V^)  ]BCWR 
m  =  current  monthly  data 

(30:25) 

EACIO  Performance  Factor  determined  by  analyst 

(l:A2-5) 

EACH  ETC  =  (1  -  CVDBCWR 

(30:26) 

EAC12  ETC  *  (1  -  (8CWP  j  -  ACWPj)/BCWP  j  )  BCWR 
BCWP  j  =.  BCWPc^^  +  BCWPc„^-i  +  BCWPc^,.j 
ACWP  J  -  ACWPc^*  +  ACWPcu^-i  +  8CWPc«,.2 

(30:26-27) 

EACH  ETC  =»  (X  -  CVW(CVI)  -►  SVW(SVI) )  BCWR 
CVW  «  Cost  Var.  Weight 
SVW  *  Schedule  Var.  Weight 


(l:A2-6) 


Lollar_^VConventional  Model" 

EAC14  *  BAC/CPI 

(32:11a) 
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Lollar  Formula 

EAC15  =  ACWP  +  (BAC-BCWP)/[SVf (SPI)  +  CVf (CPI) ] 
|SV%|  +  tCV%|  =  Total% 

SV%/Total  =  SVf 
CV%/Total  =  CVf 

(32:11) 


Parkers  Formula 

EAC16  =  ACWP  +  (BAC'BCWP)/(.3SPI  +  .7CPI) 

This  type  of  weighing  can  produce  11  different 
combinations  by  changing  by  .1  increments. 


(13:3) 


NAVSEA  Formulas  fCovach  Study) 

EAC17  «•  ACWP  +  CPI  f  Cm  (BOHR) 

EAC18  -  ACWP  +  CPI  ,j(BCWR) 

EAC19  «  ACWP  +  CPI  m(BCWR) 

EAC20  »  ACWP  +  CPI  pi5(BCWR) 

EAC21  «  ACWP  +  CPI  ,(BCWR) 

EAC22  «  ACWP  +  (CPI  ,/SPI)BCWR 
EAC23  «  CPI  m(BAC) 

EAC24  *»  BAC/SPI 
EAC2S  -  ^  P3(BAC) 

EAC26  «  CMp3{BCWR)  ACWP 
EAC27  =  CPI„(BAC) 

EAC28  -  <►  ACWP 

(12:23-24) 
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Holeman  Formula  and  Approach 

EAC29  =  ACWP  +  BCWR(PPF)  +Contract  changes  +  Sch. variations 
PPF  =  Inflation  %  +  CPI (100%)  +  Overhead  change  %/100% 

(27:22) 

EAC30  =  Range  Method 


(27:23-28) 


Totaro  Formula 

EAC31  =  ACWP  (BAC-BCWP)/PF 

PF  =  [ .25*-.25(BCWP/BAC) ]SPI  +  [ .75*+.25(BCWP/BAC) ]CPI 
♦Determined  by  analyst 

(45:31) 


ausg.S-F.<?.n!Mla 

EAC32  »  Z(BAC)*’ 

e*  »  (ACWP  /ACWP)  /{BCWPc^,^  /BCWP) 

Z  ACWP/ (BCWP)*’ 

(3:24-27) 

Automated  Contractor  Performance.. Measurement  System  (Army) 
General  formula  EAC  ACWP  +  (8AC-BCWP)P 
EAC33  P  »  1  /  CPI  j 

EAC34  P  =  1  / 

EAC3S  P  «  1  /  CPI;j 

EAC36  P  =  1  /CPI 

EAC37  P  a  Determined  by  analyst 

P  »  1/A 

A  a  (WJSPI)  +  Wj(CPI)]/Wj  Wj  w  I  +  Wj  =  100% 
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EAC38  W  1  =.5  W  2  =.5 

EAC39  W  1  =.75  W  2  =.25 

EAC40  =1.0  Wj  =0.0  same  as  P  =1/SPI 

REAC2  (See  Regression  section  this  appendix) 

REAC4  (See  Regression  section  this  appendix) 

(7:4-7) 

EAC41  =  ACWP  +  (BAC-BCWP)/P 
P  =  (CPI  6  ) (SPI) 

(7:17) 

P.^g£9.g]i\fta9Q.Aafttoja£ 

EAC42  «  ACWP  +  ETC 

ETC  «  (1  -(CVI  +CVI..4  +CVI*.,  /6)]3CW8 

EAC9, 11,13 

REACl  (See  Regression  section  this  appendix) 

(42:86-38) 

Trend  Extension 

EAC43  *  D/I  Ratio  (SPO  Direct  Coat  BAC) 

D/I  Ratio  =  Contractor  Direct  Cost  EAC/Contractor 

Indirect  and  G&A 

SPO  Direct  Cost  EAC  =  SPO  Direct  v'ost  ETC  ACWP 
SPO  Direct  Cost  ETC  =■  AETC  +  .75 (Schedule  Var.*) 

*Labor  intensive  functions  only 
AETC  «  Contractor  ETC/  (SPI) (CPI) 

93 


Contractor  ETC  =  Contractor  EAC  -  ACWP 


(38:12) 


El-Sabban  Study 

Bayesean  Probability 

E(M)  =0  M,  a/  +  Mo  +  c^  a/ 

V(m)  »  c^  /  a  a  +  (Variance) 

a  =»  (V)  (Standard  Deviation) 

M  ,  »  ACWP 
^  hhC 

C  =  M  o  /  BCWP 
O  t  .  1  M. 
a  ,  =  .05  Mo 

(20:6-8) 

Rsacaat^QiLAa'alY.^.^CBSASl 

REACl  ACV<irp  is  regressed  against  QCWP  by  least-squares- 
best-fit  to  establish  a  trend  line.  The  point  of 
vijQ  ACWP  line  where  BAG  ^  BCWP  is  the  EAC. 

REAC2  C  ■  ’ '  s.ve  C?t  ^  is  regressed  as  a  function  of 

tine.  The  final  CPI ^  is  estimated  and  multiplied  by 
the  BAC  to  determine  the  EAC. 

REAC3  The  ACWP  and  BCWP  are  both  regressed  as  a  function 

of  time.  The  time  that  corresponds  to  BCWP  =>  BAC  is 
imputed  into  the  ACWP  function  in  order  to  calculate 
the  EAC. 

(12:31) 
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Karsch  Formula 


REAC4  =  b  1  BAC“ 

Calculate  b  ^  and  bj  with  logarithm  transformation  using 
ACWP  =  bi  BCWP^*^  (Unconstrained  formula) 

Constrained  formula  b^  is  held  constant. 

Log.  transformation;  Ln  ACWP  =  Ln  b  ^  +  b2  In  BCWP 

(30:30-31) 


.SAMS.9..  EgOjula 

SAMSO  REAC2  using  only  the  current  month  and  five 
previous  months  for  least  squares  regression. 
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